← index #9015PR #5840
Related · medium · value 0.746
QUERY · ISSUE

uasynio.stream.Stream.readline fails when using SSL

openby fruchopened 2022-08-04updated 2026-03-19
bugextmod

I was trying to adapt one of the http clients (urequests/uaiohttp) to be able to use https

base on the basic example of ussl.warp_socket (and few other examples of people trying it out)
and I was hitting this failure when trying to use Stream.readline(), while it's working o.k. when using a non ssl wrapped socket

I've tested it on the unix micropython, both on 1.19 and 1.18

❯ docker run -it -w /home  -v `pwd`/lib:/root/.micropython/lib -v `pwd`:/home micropython/unix:v1.19 micropython-dev net_example.py
Address infos: [(2, 1, 6, None, bytearray(b'\x02\x00\x01\xbbh\xed\x89\xd3\x00\x00\x00\x00\x00\x00\x00\x00')), (2, 2, 17, None, bytearray(b'\x02\x00\x01\xbbh\xed\x89\xd3\x00\x00\x00\x00\x00\x00\x00\x00')), (2, 3, 0, None, bytearray(b'\x02\x00\x01\xbbh\xed\x89\xd3\x00\x00\x00\x00\x00\x00\x00\x00'))]
Connect address: bytearray(b'\x02\x00\x01\xbbh\xed\x89\xd3\x00\x00\x00\x00\x00\x00\x00\x00')
Traceback (most recent call last):
  File "net_example.py", line 53, in <module>
  File "uasyncio/core.py", line 1, in run_until_complete
  File "uasyncio/core.py", line 1, in run_until_complete
  File "uasyncio/core.py", line 1, in run_until_complete
  File "net_example.py", line 46, in main
  File "uasyncio/stream.py", line 1, in readline
TypeError: unsupported types for __iadd__: 'bytes', 'NoneType'

here's the example that reproduce it:

try:
    import usocket as _socket
except:
    import _socket
try:
    import ussl as ssl
except:
    import ssl

# https://github.com/micropython/micropython/blob/master/examples/network/http_client_ssl.py
# https://freewavesamples.com/files/Ensoniq-ZR-76-08-Dope-92.wav
# http://www-mmsp.ece.mcgill.ca/Documents/AudioFormats/WAVE/Samples/AFsp/M1F1-Alaw-AFsp.wav

import uasyncio as asyncio

from uasyncio.stream import Stream
from uasyncio import core


async def main(https=True):
    s = _socket.socket()

    if https:
        ai = _socket.getaddrinfo("freewavesamples.com", 443)
    else:
        ai = _socket.getaddrinfo("www-mmsp.ece.mcgill.ca", 80)
    print("Address infos:", ai)
    addr = ai[2][-1]

    print("Connect address:", addr)
    s.connect(addr)

    if https:
        s = ssl.wrap_socket(s)
    s.setblocking(False)
    yield core._io_queue.queue_write(s)
    ss = Stream(s)

    if https:
        ss.write(b"GET /files/Ensoniq-ZR-76-08-Dope-92.wav HTTP/1.0\r\nHost: freewavesamples.com\r\n\r\n")
    else:
        ss.write(b"GET /Documents/AudioFormats/WAVE/Samples/AFsp/M1F1-Alaw-AFsp.wav HTTP/1.0\r\nHost: www-mmsp.ece.mcgill.ca\r\n\r\n")

    await ss.drain()
    # print(await ss.read(4096)) # this works both on http/https
    print(await ss.readline()) # readline works only on http
    print(await ss.readline())

    ss.close()


loop = asyncio.get_event_loop()
loop.run_until_complete(main())
loop.close()
CANDIDATE · PULL REQUEST

moduasyncio: Add SSL support

closedby tveopened 2020-03-29updated 2023-12-13
extmod

This PR sits on top of #5819 and ~#5825~#6615 and adds SSL support to uasyncio. Update: I backed out #5819 and ~#5825~#6615 to make the PR easier to review on github, obviously CI now fails. I'll do the proper rebase once the fate of these other PRs is determined.

Adding the wrap_socket to Stream.open_connection is relatively straight-forward. It's different from CPython (no SSLContext) but compatible. The trickier part is to deal with poll, see comments in modussl_mbedtls. I believe I came up with a clean solution but so far I have failed to produce a test case that generates the problem so I can have a fail-before/success-after confirmation that the solution actually works or actually is necessary. I need to think about a test case more but if anyone can construct one I'd love that too!

I added a errstr class function to ussl in order to convert an ssl error number to a string. This is needed when using uasyncio because it uses read/write which means ussl doesn't raise an OSError (which would have the error string) and when the app gets the OSError through the Stream stuff it is again unintelligible. Now at least one can write something like if e.args[0] < -120: print(ssl.errstr(e.args[0])).

Open to-do items:

  • figure out whether the poll patch is needed and works
  • add strerr and poll patch to axtls
  • make recv and send run their OSError raising through ussl_raise_error to get an error message in there
  • add errstr to docs
  • add wrap_socket to Server.serve
  • test against esp32 espidf v3
  • test on other platforms

I should say that the tests don't all pass. Specifically, net_inet/test_tls_sites fails on the esp32. I'm pretty sure the issue is an out-of-memory in esp-idf situation. It ends up dropping wifi. I believe if I fix #5835 there may be just enough memory again to sail through...

I'll proceed once I have feedback

Keyboard

j / / n
next pair
k / / p
previous pair
1 / / h
show query pane
2 / / l
show candidate pane
c
copy suggested comment
r
toggle reasoning
g i
go to index
?
show this help
esc
close overlays

press ? or esc to close

copied