uasynio.stream.Stream.readline fails when using SSL
I was trying to adapt one of the http clients (urequests/uaiohttp) to be able to use https
base on the basic example of ussl.warp_socket (and few other examples of people trying it out)
and I was hitting this failure when trying to use Stream.readline(), while it's working o.k. when using a non ssl wrapped socket
I've tested it on the unix micropython, both on 1.19 and 1.18
❯ docker run -it -w /home -v `pwd`/lib:/root/.micropython/lib -v `pwd`:/home micropython/unix:v1.19 micropython-dev net_example.py
Address infos: [(2, 1, 6, None, bytearray(b'\x02\x00\x01\xbbh\xed\x89\xd3\x00\x00\x00\x00\x00\x00\x00\x00')), (2, 2, 17, None, bytearray(b'\x02\x00\x01\xbbh\xed\x89\xd3\x00\x00\x00\x00\x00\x00\x00\x00')), (2, 3, 0, None, bytearray(b'\x02\x00\x01\xbbh\xed\x89\xd3\x00\x00\x00\x00\x00\x00\x00\x00'))]
Connect address: bytearray(b'\x02\x00\x01\xbbh\xed\x89\xd3\x00\x00\x00\x00\x00\x00\x00\x00')
Traceback (most recent call last):
File "net_example.py", line 53, in <module>
File "uasyncio/core.py", line 1, in run_until_complete
File "uasyncio/core.py", line 1, in run_until_complete
File "uasyncio/core.py", line 1, in run_until_complete
File "net_example.py", line 46, in main
File "uasyncio/stream.py", line 1, in readline
TypeError: unsupported types for __iadd__: 'bytes', 'NoneType'
here's the example that reproduce it:
try:
import usocket as _socket
except:
import _socket
try:
import ussl as ssl
except:
import ssl
# https://github.com/micropython/micropython/blob/master/examples/network/http_client_ssl.py
# https://freewavesamples.com/files/Ensoniq-ZR-76-08-Dope-92.wav
# http://www-mmsp.ece.mcgill.ca/Documents/AudioFormats/WAVE/Samples/AFsp/M1F1-Alaw-AFsp.wav
import uasyncio as asyncio
from uasyncio.stream import Stream
from uasyncio import core
async def main(https=True):
s = _socket.socket()
if https:
ai = _socket.getaddrinfo("freewavesamples.com", 443)
else:
ai = _socket.getaddrinfo("www-mmsp.ece.mcgill.ca", 80)
print("Address infos:", ai)
addr = ai[2][-1]
print("Connect address:", addr)
s.connect(addr)
if https:
s = ssl.wrap_socket(s)
s.setblocking(False)
yield core._io_queue.queue_write(s)
ss = Stream(s)
if https:
ss.write(b"GET /files/Ensoniq-ZR-76-08-Dope-92.wav HTTP/1.0\r\nHost: freewavesamples.com\r\n\r\n")
else:
ss.write(b"GET /Documents/AudioFormats/WAVE/Samples/AFsp/M1F1-Alaw-AFsp.wav HTTP/1.0\r\nHost: www-mmsp.ece.mcgill.ca\r\n\r\n")
await ss.drain()
# print(await ss.read(4096)) # this works both on http/https
print(await ss.readline()) # readline works only on http
print(await ss.readline())
ss.close()
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
loop.close()
moduasyncio: Add SSL support
This PR sits on top of #5819 and ~#5825~#6615 and adds SSL support to uasyncio. Update: I backed out #5819 and ~#5825~#6615 to make the PR easier to review on github, obviously CI now fails. I'll do the proper rebase once the fate of these other PRs is determined.
Adding the wrap_socket to Stream.open_connection is relatively straight-forward. It's different from CPython (no SSLContext) but compatible. The trickier part is to deal with poll, see comments in modussl_mbedtls. I believe I came up with a clean solution but so far I have failed to produce a test case that generates the problem so I can have a fail-before/success-after confirmation that the solution actually works or actually is necessary. I need to think about a test case more but if anyone can construct one I'd love that too!
I added a errstr class function to ussl in order to convert an ssl error number to a string. This is needed when using uasyncio because it uses read/write which means ussl doesn't raise an OSError (which would have the error string) and when the app gets the OSError through the Stream stuff it is again unintelligible. Now at least one can write something like if e.args[0] < -120: print(ssl.errstr(e.args[0])).
Open to-do items:
- figure out whether the poll patch is needed and works
- add strerr and poll patch to axtls
- make recv and send run their OSError raising through ussl_raise_error to get an error message in there
- add errstr to docs
- add wrap_socket to Server.serve
- test against esp32 espidf v3
- test on other platforms
I should say that the tests don't all pass. Specifically, net_inet/test_tls_sites fails on the esp32. I'm pretty sure the issue is an out-of-memory in esp-idf situation. It ends up dropping wifi. I believe if I fix #5835 there may be just enough memory again to sail through...
I'll proceed once I have feedback