RFC: Webrepl feature parity with mpremote
The current webrepl-code seems to be a bit outdated lack behind the feature progress of mpremote. Additionally, the current file-transfer logic that aims to work while the REPL is blocked, however this can lead to filesystem corruption if the blocking code accesses the filesystem at the same time.
I propose the following:
- move the password-checking logic from
_webrepltowebrepl.py - remove the current file-transfer protocol, and with it the
_webreplmodule. - move the repl interaction from websocket "text" packages to "binary" packages, as the output from the repl is not required to be valid UTF-8, but just a byte sequence.
- implement the same logic that
mpremoteuses for file transfers in the webrepl client.
Removing _webrepl and moving the password logic to webrepl.py even seems to make the firmware smaller:
text data bss dec hex filename
606100 0 381308 987408 f1110 /home/jenkins/tmp/micropython/ports/rp2/build-RPI_PICO_W/firmware.elf
605668 0 381292 986960 f0f50 /home/jenkins/tmp/micropython/ports/rp2/build-RPI_PICO_W/firmware.elf
I've implemented a proof-of-concept here: https://github.com/felixdoerre/webreplv2 together with a new webrepl client.
The benefits over the current webrepl are:
- Allow binary data transfers over the websocket (this also helps other usages of the webrepl, like Thonny, or using the websocket as backend for
mpremoteinstead of a serial device) - Implement a File-Browser in the webrepl-client, with file up- and download using the same primitives as
mpremote - Implement
mipto install packages directly from the webrepl-client - Implement the filesystem-mounting feature
- Implement direct mounting of a local directory through the browser (without uploading). Due to webapi limitations this is only a readonly mount.
- Allow browser-saving of the webrepl password
- Auto-connect the webrepl, if the target (and password) is known.
Do you want to include this into micropython, replacing the current webrepl module?
webrepl: Changes for more webrepl features while making it smaller.
This change:
- Moves the password checking to python
- Removes the special file transfer protocol (This changes allows to extend
mpremoteto support webrepl just like a serial port) - Moves the REPL data to websocket binary packages
The change should be compatible with the current micropython version, however a new webrepl client needs to be deployed under https://micropython.org/webrepl/
Currently, to help you testing, I've adjusted the default URL to https://felix.dogcraft.de/webrepl/, where I host the corresponding draft "new" webrepl client. I've pushed the modified js code here, if you want to take a look: https://github.com/felixdoerre/webreplv2. I'm not completely sure, if I should open a PR in https://github.com/micropython/webrepl, as this seems to be the authoritative source of the client, but the repo seems to be abandoned for some time now.
The new webrepl client currently features (See also #13540):
- A file browser, allowing directory listings, downloads and uploads. This uses the same injected code as
mpremote. - An implementation of the
mount-feature frommpremotethat works with the same injected code. It allows mounting files from the browser's local storage (which can host some python scripts that one wants to bring for debugging), or from a local folder, that is drag-and-dropped into the webrepl (read-only), this is useful for rapid development/testing. - An implementation of mip with a package browser to list and download packages directly from https://micropython.org/pi/
Currently the "mount" feature has to be activated "manually" (by calling hookme() from the developer tools, and then executing __mount() in the repl manually). Also the UI for the file browser might not be super intuitive yet (the + for upload and mounting a local folder might be too small and not clear enough). So I'm hoping for UI improvement suggestions.
If you want, I can split the mount and the mip implementation into separate PRs, but the file-browser and this PR are dependent on each other.
With this change the _webrepl module from micropython is not needed anymore and can be removed.
@felixdoerre this could be a good opportunity
to add SSL support to WebREPL using
SSLContextsee https://github.com/micropython/micropython/pull/5611#issuecomment-1314686679 👍🏼I've pushed a few adjustments and an (optional) bonus commit to allow specifying an ssl context.