This may or may not be related, but main() is meant to be a tight loop and since I can't say for sure how long ingest() will take I really need to call it via schedule(); when I do that I can only get one buffer's worth of data before reading from SERIAL_TWO fails like I hit EOF. I know there's more there but I can't get to it, could this also be some sort of schedule()-induced deadlock?
Cannot init USB devices in boot.py
I tested this on a pico running MP 1.23 and the latest release of usb-device-cdc.
boot.py:
import usb.device
from usb.device.cdc import CDCInterface
IFACE_TWO = CDCInterface()
IFACE_TWO.init(timeout=0)
#usb.device.get().init(IFACE_TWO, builtin_driver=True)
main.py:
from time import sleep_ms
def main():
while True:
print(IFACE_TWO)
sleep_ms(1000)
if __name__ == "__main__": main()
Running this as-is, IFACE_TWO does show up in main.py, but uncommenting the init function causes the process to error out. This discord post suggests you can call usb.device.get.init() in boot.py, is this no longer true?
Serial connections dropping characters, not flushing buffers
I tested this on a Pico running MP 1.23 and the latest version of usb-device-cdc on mip
main.py:
from io import BytesIO
from select import poll, POLLIN, POLLHUP, POLLERR
from time import sleep_ms
import usb.device
from usb.device.cdc import CDCInterface
SERIAL_TWO = None
def ingest():
f = BytesIO(512)
ba = bytearray(256)
mv = memoryview(ba)
print("beginning ingest")
# SERIAL_TWO.read(4)
while True:
bytes_read = SERIAL_TWO.readinto(mv)
if bytes_read:
f.write(mv[:bytes_read])
print("{} written, total {}".format(bytes_read, f.tell()))
else:
break
f.seek(0)
print(f.read(-1))
def main():
global SERIAL_TWO
buf = bytearray(4)
SERIAL_TWO = CDCInterface(timeout=0)
usb.device.get().init(SERIAL_TWO, builtin_driver=True)
while not SERIAL_TWO.is_open():
sleep_ms(100)
api_poll = poll()
api_poll.register(SERIAL_TWO, POLLIN)
while True:
cmd_waiting = api_poll.poll(0)
if cmd_waiting and cmd_waiting[0][1] is POLLIN:
# SERIAL_TWO.read(4)
ingest()
sleep_ms(1000)
if __name__ == '__main__': main()
client.py:
from serial import Serial
from string import ascii_letters
def main():
test_str = ascii_letters * 7
conn = Serial("/dev/ttyACM1", 115200)
conn.read_all()
data = "\x03" + "\n" + "\x21" + "\n" + test_str
conn.write(data.encode())
if __name__ == '__main__': main()
I need to read four bytes off the input to confirm I should call ingest(), but in doing so I consistently lose eight bytes after the first call to readinto() (the output skips V -> e after 256 bytes) and some of the content never appears to leave the buffer. Moving the read() call into ingest() doesn't help, nor does using readinto() instead.
There is a bug here, thanks for the report. When N bytes was read from the CDC interface, N < 64, and the host has more than N bytes still to send then (64-N) bytes were dropped. Fix incoming.
This may or may not be related, but main() is meant to be a tight loop and since I can't say for sure how long ingest() will take I really need to call it via schedule(); when I do that I can only get one buffer's worth of data before reading from SERIAL_TWO fails like I hit EOF. I know there's more there but I can't get to it, could this also be some sort of schedule()-induced deadlock?
This part has the same root cause as #873 - the callbacks for completed USB transfers run using the same mechanism as schedule() so you can't block waiting for USB data inside a scheduled callback, it will block indefinitely. Need to find a way to exit the callback to allow more USB data transfer.
I will also look at adding a cdc.any() function that returns the number of buffered bytes. That would make it possible to delay calling the callback again until you know all the bytes are ready. Can also look at putting together an asyncio example for CDC (I believe it shouldn't need any code changes) as this type of application is a lot easier to structure with asyncio.
Please give some more details about this means: what exactly do you do, and what error output do you see?
A soft reset (i.e. Ctrl-D in the REPL) will still disconnect the USB port each time, because the runtime USB interface objects have to all be freed and recreated.
A hard reset (i.e. plugging the rp2 in, or calling
machine.reset()) should come up immediately with the two CDC ports.The only difference between boot.py and main.py is that the first time the device boots from a hard reset, it runs boot.py before it initialises USB. If you put the code in main.py instead of boot.py then the rp2 would first try create a device using the default USB interface, then immediately switch to the custom USB interface. This can show up on the host as connect/disconnect/connect cycle, or as an error depending on the timing.
With the init line commented out in boot.py:
mpremote fs cp boot.py :mpremote run main.pyI see the print statements I expect:
With the init line uncommented in boot.py:
mpremote fs cp boot.py :mpremote run main.pyI do not see the print statements I expect:
I get dropped back to the terminal, and connecting with
mpremote a0yields no output. Attempting to replace boot.py even after unplugging/replugging the pico also yields the above OSError; I have to drop to a shell and useos.unlink().Ah snap,
mpremote rundoes a soft reset before it runs the provided file. Because boot.py has loaded the custom USB interface, it disconnects the USB port every time (then boot.py runs and adds it back, so you end up where you started.)If you copy both boot.py and main.py to the filesystem then main.py works and you can use
mpremote a0to view the output, but you still can't usempremote runany more.I think the fix for this will be the planned work to have mpremote detect when the USB port is in a disconnect/reconnect cycle, and recover from it instead of erroring out.
Is there a timeline for when the planned work might be finished?
Not at the moment, I'm afraid.
I assume this is impacting
mpremote connectas well (also including a soft reset)? Running similar as the OP above, I'm finding that I can connect directly to REPL by connecting to its COM port - but not usingmpremote connect.Not sure if anything has changed since the last update here - though revisiting this a year later, now on MicroPython v1.27 - everything seems to be working, and I'm not sure I can find a way to make it not?
I'm seeing success with both (separately) MicroPico 4.3.4 in VS Code, and mpremote 1.27.0.
Curious if anyone else here still watching can help test / validate? Thanks!
Though for reasons I don't yet fully understand, having
usb.device.get().init(...)included as part ofboot.pyinterferes with running "MicroPico: Run current file on Pico" (micropico.run) from MicroPico in VS Code.Disabling "Micropico: No Soft Reset On Run" (
micropico.noSoftResetOnRun) seems to then resolve this.Though then, and I suspect unrelated, I get into trouble if I try using Ctrl+C to quit a loop - same as reported in https://github.com/paulober/MicroPico/issues/294 - though pretty sure this is an issue with MicroPico and not micropython.