QUERY · ISSUE

urequests still using HTTP/1.0

openby linusoftopened 2024-04-11updated 2024-05-06

The urequests module is still using HTTP/1.0 for HTTP requests:

line 39 in micropython-lib/micropython/urllib.urequest/urllib/urequest.py on the master branch:

s.write(b" HTTP/1.0\r\nHost: ")

Since a host header is included, can this be updated to use HTTP/1.1 or even HTTP/2.0?

Some sites (e.g. mongodb) are now responding with '426 Update Required' for HTTP/1.0 requests.

Thanks

3 comments

th4ruka · 2024-05-05

Hey! I also think it is better to update the header in line 39 of micropython-lib/micropython/urllib.urequest/urllib/urequest.py as,

- s.write(b" HTTP/1.0\r\nHost: ")
+ s.write(b" HTTP/2.0\r\nHost: ")

As I know, most major web servers and CDNs provide backward compatibility. Therefore there will be no harm by updating it.

sosi-deadeye · 2024-05-05

HTTP/2 is a binary protocol and uses HPACK for header compression. Changing this, require additional code to handle the protocol and header compression.

I also think the change to HTTP/1.1 may require some additional code, to handle the different connection types : close, keep-alive.

In addition, there are different Transfer-Endocings.

mattytrentini · 2024-05-06

It would be good to merge back some of the changes from the CircuitPython requests library (they've updated it since originally starting with the MicroPython implementation). In it, they respond with v1.1, though I'm not sure it 100% conforms to spec...

https://github.com/adafruit/Adafruit_CircuitPython_Requests/blob/main/adafruit_requests.py

CANDIDATE · PULL REQUEST

urequests support for user-provided http versions

closedby coldenateopened 2023-01-01updated 2023-03-13

About

Hi there! :)
This pull request implements a simple update to the urequests's request function. This implementation includes an argument to allow for modifcation of the socket's written http version.

Why this was done?

I made this change in my own fork because I had scrapped together an API in FastAPI, and this did not accept my microcontroller's HTTP 1.0 post requests.

Please let me know if there is a reason the HTTP 1.1 version remains in the code - despite possible need for it to be http 1.0. I am new to learning about HTTP and related topics :)

Cheers!

3 comments

jimmo · 2023-01-16

Please let me know if there is a reason the HTTP 1.1 version remains in the code - despite possible need for it to be http 1.0. I am new to learning about HTTP and related topics :)

@coldenate The simple answer is that this is only the second time (to my knowledge) that someone has reported a server not supporting HTTP/1.0 (which is a surprising thing for a web server to do), whereas we have no idea how many people are using urequests with servers that only support HTTP/1.0 (odd, but reasonable).

Your suggestion to make it optional is probably reasonable. I would argue that perhaps we are de-facto using HTTP/1.1 anyway (setting Host: and Transfer-Encoding:), so possibly should just make the change rather than just making the code bigger. (And also, seriously, how many web servers are there out there that don't support HTTP/1.1 ???).

I'd be interested to see if anyone else has any comments.

samveen · 2023-03-05

I made this change in my own fork because I had scrapped together an API in FastAPI, and this did not accept my microcontroller's HTTP 1.0 post requests.

@coldenate Nathan, would you confirm that the server implementation is correct? In my rather limited experience, I've found that most cases of clients failing to handle HTTP/1.0 responses correctly are infact badly coded servers (at least my implementations of servers had that issue, which has lead to #623 ).

coldenate · 2023-03-05

@samveen It most likely is my poorly coded server 🥲, but I will look further into it. I was using FastAPI to make the server.

urequests still using HTTP/1.0

urequests support for user-provided http versions

About

Why this was done?

Keyboard