UART.readline() reads the buffer even when it does not end in a newline character
The UART documentation says:
UART.readline()
Read a line, ending in a newline character.
Return value: the line read or None on timeout.
If I connect a wire between TX2 and RX2 on the ESP32, and go:
from machine import UART
import time
u = UART(2)
s = u.write("begin1 end1\nbegin2 ");
# s = u.write("end2\nbegin3 ");
time.sleep(0.1)
print(u.any(), u.readline())
print(u.any(), u.readline())
print(u.any(), u.readline())
print(u.any(), u.readline())
The output comes out as:
19 b'begin1 end1\n'
7 b'begin2 '
None
None
You can uncomment the second u.write() to prove that the strings will be concatenated in the buffer prior to the readline().
(The timeout value in the constructor is not documented, but I've worked out that it is in milliseconds.)
With quite a bit of fiddling I can prove the same effect on the ESP8266 with the following code loaded into main.py:
from machine import UART
import time, os
u2 = UART(1, baudrate=115200)
print("startup")
u2.write("startup\n")
time.sleep(3) # allow for interrupt
print("3seconds")
u2.write("3seconds\n")
os.dupterm(None, 1)
u = UART(0, baudrate=115200, rxbuf=240)
s = u.write("begin1 end1\nbegin2 ");
s = u.write("end2\nbegin3 ");
time.sleep(0.1)
for i in range(5):
u2.write("%d %d %s\r\n" % (i, u.any(), str([u.readline()])))
Then you can monitor the output of the TX1 from the ESP8266 (which goes out of the GPIO2) by connecting it to RX2 of a spare ESP32 which is running the program:
Then on the ESP32 you run:
from machine import UART
u = UART(2, timeout=5000)
while True:
print(u.readline())
(Don't forget to connect the RX to TX of the ESP32 and the VIN and G pins from the ESP32 to the ESP8266 to send it the power.)
The result is:
b'...\x02startup\n'
b"0 1 [b'3seconds\\r\\n']\r\n"
b"1 1 [b'begin1 end1\\n']\r\n"
b"2 1 [b'begin2 end2\\n']\r\n"
b"3 1 [b'begin3 ']\r\n"
b'4 0 [None]\r\n'
The fact that UART.any() can return 1 even when there are >1 characters in the buffer is at least documented. However, I can't tell how to use select.poll() for a sophisticated way of querying the available characters.
Normally I'm not daft enough to waste so much time trying to use UART0 on the ESP8266 (owing to the problem that it blocks the Repl), but I'm trying to do something with a Wemos Arduino Megas that -- with the right selection of dip-switches -- wires the atmega to the esp8266 on its USB0, so I have no choice.
Now that I have identified this issue (which was very hard to see when you can't print debug messages on a repl blocked esp8266), I can make some code to concatenate the incomplete lines. But it would be much more useful if this was done properly inside the UART buffer itself.
stmhal: UART(..., timeout=0) is broken for stream protocol
UART(..., timeout=0) is supposed to put UART in fully non-blocking mode, where operations on byte strings either complete almost immediately (possibly partially), or fail with EAGAIN. This semantics most realistically requires buffering.
Currently, stmhal with timeout=0 doesn't behave reasonably: .write() always fails with ETIMEDOUT, even though it actually transmits 1st byte. .read() returns only first available byte, and appear to lost next bytes. Issue with .write() is due to fact that its implemented without buffering, and timeout is just passed to HAL_UART_Transmit() which has rather different semantics for timeout: https://github.com/micropython/micropython/blob/9e0a3d46b6eb176a3450d565c9e172eb22f9c8dc/stmhal/hal/f2/src/stm32f2xx_hal_uart.c#L622
Don't know how to explain issues with .read(), but:
>>> import pyb
>>> u = pyb.UART(4, 115200, timeout=0)
# Connect in another window to that uart with picocom, type "123"
>>> print(u.read(3))
b'1'
>>> print(u.read(3))
None
For comparison:
>>> u = pyb.UART(4, 115200, timeout=1)
# Connect in another window to that uart with picocom, type "123"
>>> print(u.read(3))
b'123'
>>> print(u.read(3))
None
So, as proper fix will require a lot of rework, I propose to work that around by checking for timeout=0 in constructor and replacing it with timeout=1.