mpremote should let user control DTR and RTS signals
I'm operating on ESP32-CAM with ESP32-CAM-MB programmer / usb serial converter
to avoid continuous hard reboots, I have to disable DTR and RTS signals, here you can find some technical details:
https://arduino.stackexchange.com/questions/90343/esp32-cam-and-mb-code-uploaded-but-nothing-in-serial-monitor
only some serial terminals can be configured to avoid this problem, for example:
picocom
picocom --baud 115200 --lower-rts --lower-dtr /dev/ttyUSB0
thonny (configuration.ini)
[ESP32]
port = auto
interrupt_on_connect = True
sync_time = True
local_rtc = True
restart_interpreter_before_run = True
dtr = False
rts = False
without this option, mpremote wont let me operate with this board
mpremote: Workaround ESP reset quirk at disconnect time.
Summary
This is a possible alternative fix for #9659 which uses a different approach to #17776 and #17800.
Background
There are two conflicting problems with use of DTR and RTS in mpremote (and other serial host programs):
-
USB-CDC implementations tend to use "DTR is set" as a signal for "host is connected" (a reasonable thing to do), and therefore don't send any data to the host if DTR is cleared.
-
ESP8266 and ESP32 boards use a convention where setting
(DTR&&!RTS)means "put IO0 low for bootloader mode" and(!DTR&&RTS)means "trigger reset", allowing the host to reset the chip to bootloader mode. Traditionally this is implemented in a small circuit - the circuit mostly exists so that default behaviour of an open serial port(DTR&&RTS)doesn't trigger a reset.With the Espressif integrated "USB Serial & JTAG" peripheral and Espressif TinyUSB native USB stack the same reset logic is implemented by looking for a sequence of line state transition packets (implemented in hardware and software, respectively).
#9659 describes a problem where Windows (and/or pyserial on Windows) clears DTR before RTS when closing the port, and therefore triggers a hard reset each time mpremote exits.
@Josverl has done a lot of hard work and testing to find a DTR & RTS setting which works for (2) without causing problems due to (1), and has come up with a heuristic for detecting possible Espressif boards. The challenge is that pretty much any USB/Serial chip on the market has been connected to an ESP8266 or ESP32 at some point, and there's no way to tell what MCU is connected from the USB side.
There also cases like #17999 where an Espressif chip with a native USB implementation needs (1) in order to function correctly.
Approach in this PR
Instead, we can manually clear RTS before DTR when closing the port, to avoid the reset issue. Opening the port can mostly use the default behaviour (RTS & DTR both set). So far have only found one exception: on Windows the Silicon Labs CP210x driver toggles DTR and RTS with a long delay in between the first time the port is opened after a reset, which triggers a hard reset the first time mpremote tries to connect - have worked around this specific case on connection.
In mpremote, the transport close() is called from a finally block in the mpremote main module (via do_disconnect()) - so this should always happen provided the Python process isn't terminated by the OS.
Testing
On Linux & WSL my test process is:
mpremote a0 exec "a=53"; mpremote a0 resume exec "print(a)"
If the resume works correctly then this should print 53. If a reset occurs when mpremote exits then it will print NameError: name 'a' isn't defined.
Windows allocates a new COM port for each device, so made a batch file to only type the port name once:
python -m mpremote connect %1 exec "a=52"
python -m mpremote connect %1 resume exec "print(a)"
Results:
| Board & Port | USB Device | Linux | Windows 11 | WSL | MacOS |
|---|---|---|---|---|---|
| ESP32-S3-DevKitC (USB port) | MP TinyUSB | OK | OK | OK | |
| ESP32-S3-DevKitC (UART port) | SiLabs CP2102N | OK | OK | OK | |
| ESP32-C6-DevKitM (USB port) | Espressif Serial/JTAG Peripheral | OK | OK | OK | |
| ESP32-C6-DevKitM (UART port) | SiLabs CP2102N | OK | OK | OK | |
| ESP32-C2-DevKitM | SiLabs CP2102N | OK | OK | OK | |
| ESP32 Pico Core Board V3 | SiLabs CP2102 | OK | OK | OK | |
| ESP32 ECO1 Core Board | SiLabs CP2102 | OK | OK | ||
| Seeed XIAO (ESP32-C3) | Espressif Serial/JTAG Peripheral | OK | OK | OK | |
| ESP32-WROVER-KIT | FTDI FT2232H | OK | OK | ||
| ESP8266 NodeMCU Amica | SiLabs CP2102 | OK | OK | ||
| ESP8266 WeMos D1 Mini | WCH CH340 | OK | OK | OK | |
| ESP8266 WeMos D1 | WCH CH340 | OK | OK | ||
| RPI_PICO | MP TinyUSB | OK | OK | OK | |
| Seeed WIO Terminal (SAMD D51) | MP TinyUSB | OK | OK | ||
| PyBoard SF-2W | MP stm32 USB-CDC | OK | OK | ||
| NUCLEO-H723G CN1 | st-link embedded USB-CDC adapter | OK | OK |
- Windows tests are using a VM, but the USB host controller PCI device is passed through to the VM with VFIO so the USB device only interacts with Windows, should be equivalent to a native host. (Passthrough of the USB controller is necessary to reproduce the quirk with CP210x driver after power-on, if the USB device is first enumerated by Linux then the problem goes away.)
- I haven't tested every board on WSL (using usbipd-win) but tested one of each chip at least.
- If anyone has ESP32 boards with WCH USB/serial chips onboard then I'd appreciate any testing you could do.
- I don't have access to a Mac to verify there's no regression there, but I don't expect one (the port opening sequence hasn't changed for macOS, and clearing RTS early shouldn't break anything).
- Also did some manual tests of pressing the Reset button while mpremote REPL was connected to double check the board doesn't reset into bootloader mode.
Trade-offs and Alternatives
- If the mpremote process is killed by Windows then Windows may clear DTR before RTS during cleanup and trigger a hard reset (not verified), but this seems like a reasonable limitation. The close() path is in a finally block otherwise, so any "normal" mpremote exit should hit it.
- We could keep adding quirks for specific USB IDs, but I think this risks getting into a game of "whack a mole" with vendor USB/serial chip choices.
Follow Up Work
- We should add command line options to manually set DTR and RTS on the serial port. This will allow using Espressif devices where DTR and RTS are wired directly to the control pins, and provide a way to work around any other unusual serial configs. Can be done in a follow-up PR.