← index #251PR #600
Duplicate · high · value 3.223
QUERY · ISSUE

urequests: miscalculation of Content-Length when dealling with unicode data

openby 7th-heavenopened 2018-01-13updated 2025-08-26
unicode

code in urequests ver 0.5.1:
s.write(b"Content-Length: %d\r\n" % len(data))

"len" calcs a single unicode as 1 length.
to correct it. encode the data before calcs the length
as: s.write(b"Content-Length: %d\r\n" % len(data.encode('')))
plz fix it.

6 comments
pfalcon · 2018-01-14

Perhaps, you should call .encode() on your side instead?

7th-heaven · 2018-01-14

Yup, it's works correctly in micorpython 1.9.3 on nodemcu with eps8266. And ascii data works well too.

Before I fix this, There's no problem when dealling with ascii datas.
But, as I post the unicode data to WebHook on the server via https.The server replies 'illegal parameters'.

pfalcon · 2018-01-14

If it wasn't clear, you should call .encode() method in your application, and pass bytes as "data" param to urequests. You can see this requirement in the upstream requests module too: http://docs.python-requests.org/en/master/api/#requests.request

(This issue can remain open. If there will be a dozen more similar reports from users, the issue can be reconsidered.)

7th-heaven · 2018-01-14
def post_data(data):
    url='WEBHOOKURL'

    jsonmsg = {
        "msgtype": "text",
        "text": {
            "content":   data
        },
        "at": {
        "atMobiles": 
          [],
        "isAtAll": False
              }
        }     
    jsonheader = {
        'Content-Type': 'application/json'
    }
    r = urequests.post(url, data=json.dumps(jsonmsg), headers=jsonheader)
    return r

This is the part of my app.
This function posts unicode text to server, and then the server send the unicode text to the clients.
the unicode text can not call .encode() before POST.

For exsamples:

  1. I called post_data('测试') , in requests-lib of python3 on PC, it's worked succesfully.
    BUT in urequests-lib of micropython on nodemcu with esp8266, IT DO NOT WORKS .
    the server replies 'illegal parameters' to me, and clients received nothing.
    It's casued by wrong Content-Length.

  2. I called post_data(str('测试').encode('')) , the server replies 'OK' to me, clients received 'æµ^Kè¯'
    Oh, it's works. but gibberish recevied.

I'v tried data = None, headers = None, json = jsonmsg. The resluts are the same as exsamples.
Is the wrong way I tried?

kenjican · 2019-01-13

@7th-heaven I encountered same issue,, this is my solution:
data should be encode as bytes and pass headers parameter:
res = urequests.post(url, data=(ujson.dumps(dd['msg'])).encode(), headers={'Content-Type':'application/json'})

It looks like DingTalk's API....I've tested and proven that works

projectgus · 2025-08-26

This is very similar to #746 which is a report about the same behaviour with umqtt.

Have the same suggestion here, that silently truncating utf-8 arguments is probably not a good choice - it should either explicitly fail, or attempt to encode even if this does cost some RAM (and the documentation can recommend passing bytes to save RAM).

CANDIDATE · PULL REQUEST

urequests: Fixed UTF-8 support for request data

openby asid21opened 2023-01-28updated 2025-08-26
unicode

UTF-8 support for request data was added, it will be able to handle special characters correctly.

1 comment
foxt · 2023-09-16

+1.
Reporting temperature data to Home Assistant requires the °C to be included.

A quick work around I found to work is just

urequests.post(
    ...
   data=json.dumps({ ... }).encode()
)

Keyboard

j / / n
next pair
k / / p
previous pair
1 / / h
show query pane
2 / / l
show candidate pane
c
copy suggested comment
r
toggle reasoning
g i
go to index
?
show this help
esc
close overlays

press ? or esc to close

copied