Replies: 6 comments 29 replies
-
|
I may have misunderstood, but judging by the image having the stubs folders marked as source root, I got the impression that the proposed solution entails uploading stubs directly onto the device filesystem. If that is the case, I would avoid doing so. Microcontrollers are by definitions working with restricted resources, and any amount of free storage space is valuable, this is especially true for some ESP8266 devices. If I have indeed misunderstood or if there is a way to do what you propose without uploading stubs on the device file system, then I believe that this is actually the best practice - involving the user in selecting the stub device type. There are a few edge case scenarios to handle that come to mind. The plugin probably should allow the user to somehow manually select/search for the specific device type, if for whatever reason the detection fails or the user has some kind of alternative intent. This could be handled together with manually selecting one's own local stubs. Aside from uploading stubs directly onto the device, which might just be my misunderstanding, this solution seems like the best way to handle stubs in this plugin. |
Beta Was this translation helpful? Give feedback.
-
To me using the mark directory functionality for determining what gets uploaded to my device quite intuitive. The project I work on has two folders marked as source folders to make things more readable and organized, and with how the plugin treats them their contents get uploaded to the device file system without the containing folders, allowing me to organize both the Project and File System layout more freely. If there was just a single pre-generated folder that would be meant to store all files and folders that are meant to be uploaded, it would give users less freedom as far as organizing their projects go. Maybe it's sufficient to only exclude files like |
Beta Was this translation helpful? Give feedback.
-
|
Excluded folder can't contain any stubs. The folder is completely ignored, and the language engine just ignores it. Having several source folders is totally fine. |
Beta Was this translation helpful? Give feedback.
-
I understand now. What you propose makes sense, I can't think of a better way to handle this. |
Beta Was this translation helpful? Give feedback.
-
|
Formalized in #338 |
Beta Was this translation helpful? Give feedback.
-
I had an idea for adding more upload options while testing the pre-alpha 12 release. Currently you have the option to "Upload File/Directory to MP device". Well, for directories you could have more options such as upload entire file/upload contents. I can, however, see how this might be overcomplicating things with how the current uploading priority system with source folders works. I just thought I'd leave this here as an idea in case it could be useful to you somehow. |
Beta Was this translation helpful? Give feedback.
-
|
There is an alternative approach to auto-detection compared to the old one. I personally go back and forth between multiple ESP32 devices and USB ports when working with MicroPython. Different USB hub at work, different USB hub at home and even switching to a different USB port on my computer changes the device path of the Microcontroller I am working with. Perhaps a different form of auto-detection could be implemented in the new plugin. It would make things a tad bit easier if there was a way to track one device even after it pops up under a different device path. Known devices could be saved as device presets, or just the last connected device could be remembered and searched for in the available device paths. It doesn't really matter how complex the functionality built around this device tracking would be. Something like this could theoretically even be used to narrow down the USB serial port selection drop-down list to just devices that can somehow be verified as having MicroPython installed. Now implementing this is where things can get more difficult. Telling apart devices which have MicroPython installed is possible. The unique ID can be used and relied upon on most if not all device types. So some combination of the USB VID:PID and unique ID could be used. I can, however, see how scanning all serial port paths might not be practical. I will leave the question of whether or not this is plausible up to you, you are better suited to answer it than me. But talking about autodetect approach this is what comes to mind for me. If this would indeed be possible and wouldn't require a terribly overcomplicated solution, then some really cool features could be built around it. |
Beta Was this translation helpful? Give feedback.
-
At home it can be 2-3, and at work it is similar most of the time, max could be 5, but that is very rare. |
Beta Was this translation helpful? Give feedback.
-
|
I have reports of users having in excess 32 serial ports connected in their setup and have needed to adapt mpflash for that. It is possible to interrogate the micropython port/boart/variant and version of connected devices , but it does interrupt any running code. |
Beta Was this translation helpful? Give feedback.
-
|
That's a lot... |
Beta Was this translation helpful? Give feedback.
-
|
This is the MicroPython code that mpflash uses to get the info. Tested from v1.9.3 ... v1.23.0 |
Beta Was this translation helpful? Give feedback.
-
|
I'd rather use mpflash as is. |
Beta Was this translation helpful? Give feedback.
-
|
If there are plans to take this approach ,
I have a preference for MIT, mostly to avoid carrying too many different license types into the micropython repo. If that is not possible, then most likely I'll go back to the the somewhat older version from https://github.com/hlovatt/PyBoardTypeshed instead. But before that we need to discuss that with the MicroPython core team as well. Also see: Josverl/micropython-stubs#762 |
Beta Was this translation helpful? Give feedback.
-
something like or it might be
The license holder is JetBrains, so we need to ask our lawyers first |
Beta Was this translation helpful? Give feedback.
-
|
Understood, I'm also not a lawyer
For the stub identification / mpflash suggestion
[
{
"serialport": "COM14",
"connected": true,
"family": "micropython",
"description": "Generic ESP32 module with ESP32",
"version": "1.23.0",
"port": "esp32",
"board": "ESP32_GENERIC",
"cpu": "ESP32",
"arch": "xtensawin",
"mpy": "v6.3",
"build": "",
"location": "1-6.1",
"stub-package": "micropython-esp32-stubs==1.23.0.*",
"toml": {}
}
]Would that work , or would you also want an option to get a list of all published stubs ? |
Beta Was this translation helpful? Give feedback.
-
|
Short update: But there is additional work to be done to to get that percolated down to the individual port and board stub-packages, so ill continue down that path. |
Beta Was this translation helpful? Give feedback.
-
Full list is probably better.
Good job!
Does it mean you do not need them under MIT anymore? |
Beta Was this translation helpful? Give feedback.
-
|
Beta Was this translation helpful? Give feedback.
-
|
Hey @elmot could I ask about the reason you want to switch to using mpremote? What are the benefits over the existing Kotlin implementation? |
Beta Was this translation helpful? Give feedback.
-
I've thought of a different approach, which shaves off an incredible amount of time and avoids interrogating the device for a hash for every single file uplaod - I already have it integrated in my version of this plugin and it's been tested by both me and my colleagues. Currently it uses SHA256 for hash comparisons, but my plan is to use FNV1A32 which I think should be pretty easy to implement in a few lines - meaning it should run on any micropython port. I can create a pull request of this script and another upload related functionality that I have created if you'd be interested. It reads in chunks into a bytearray and it has handling for cases where the necessary libraries aren't available - it will simply upload everything as if no matching files were found. And regarding not touching the existing Kotlin implementation - I would like you to actually reconsider this, as there are existing bugs in the communication layer that are quite annoying - they can be mitigated by simply retrying, but there are several errors in the serial communication. Sometimes the plugin times out on "import os,errno" command, which is a part of your blindExecute method, similarly sometimes the other upload commands in this method time out as well. And finally, there seems to be some issue with frequent doBlindExecute call. Back when I submitted my very basic CRC32 hash comparison method - I though there must be something wrong with it, because it caused super frequent timeout errors with this plugin, but it seems to me that there is actually some problem with calling blindExecute very frequently. When I made the CRC hash calculation script run before file uploads on ESP32 S3 devices, it would freeze and time out in 3-8 uploaded files. My knowledge of Kotlin and its serial communication layer means I can't really contribute a fix to the serial communication problem - I'd like to learn about it, but currently I feel like I can contribute more by focusing on other things. Regarding my custom script - I am still improving it and I do plan on discussing a potential merge into MicroPython with the devs, but I first want to finish the FNV1A32 implementation and run some experiments for using very large chunks safely by utilizing bytearrays. |
Beta Was this translation helpful? Give feedback.
-
|
There is already a crc32 calculator at mpy side, as far as I understand it native code, so should work really fast. It looks like it may work in by chunk, see binascii.crc32(data[, value]) |
Beta Was this translation helpful? Give feedback.
-
The problem is that this method is not included in the ESP8266 port, at least from what I understand - hence why I want to implement FNV1A32 |
Beta Was this translation helpful? Give feedback.
-
|
Yes, CRC32 is disabled on the ESP8266 (presumably to conserve the limited flash space). Note that Brian Pugh already has an FNV1A32 implementation written as a native module...though it sounds like you're after a pure Python implementation? |
Beta Was this translation helpful? Give feedback.
-
Brian Pugh is actually the one who gave me this idea in the MicroPython discord and yes, I'm after a pure Python implementation. |
Beta Was this translation helpful? Give feedback.
-
|
In the last few days I've been considering how stubs and device type detection could be handled. Firstly, I would welcome moving the port selector drop down left of the connect/disconnect button in the MicroPython plugin window. This would make it easier to work with, as I often find myself having to go to the settings multiple times in a day, which can get annoying. I understand that this could be difficult and that it has plenty of considerations, here are a few problems and their solutions I thought of:
For both of these problems I would go with an approach similar to how thonny does it. In Thonny you have the bottom right serial port drop down, which is the default quick access. Then its interpreter settings allow you to select a device manufacturer, specific communication type and then either the serial device or WebREPL credentials. The way this then propagates is that the main quick access serial device drop down is switched to displaying This plugin's implementation could be expanded for choosing the MicroPython version as well, so that users have more control over specific stub settings should they need it. |
Beta Was this translation helpful? Give feedback.
-
That is a big topic, let's move that to another discussion. |
Beta Was this translation helpful? Give feedback.
-
|
I have a question regarding two functions that could be added to the plugin.
|
Beta Was this translation helpful? Give feedback.
-
Sure, you are welcome. That's useful addition
Sure, but better to start with a discussion |
Beta Was this translation helpful? Give feedback.
-
The problem:
There are numerous Micropython boards nowadays, and a number of Micropython forks. There is a number of additional libraries to be installed.
As a result, now it's virtually impossible to maintain a comprehensive set of python stubs inside the plugin. Additionally, that diversity of boards makes board detection by USB VID:PID useless in many cases.
Proposed solution
os.uname()pipinto the project structureAs the result, a typical project layout with `main.py` and compiled `ssd1306` library would look like:
Device layout looks like
Any thoughts or concerns?
Related to #220, #240, #141, #114, #326, #319, #316, and many others
Beta Was this translation helpful? Give feedback.
All reactions