QUERY · ISSUE

json lib module, syntax error

openby permaBoxopened 2023-11-21updated 2024-09-03

needs-info

The json lib module seems to have an issue. That was not observed in 2018.

To reproduce the error take:
r = requests.get('https://aa.usno.navy.mil/api/rstt/oneday?date=2023-11-21 &coords=48.441832,9.890139 &tz=2').json()

Generated error message:
File "requests/init.py", line 33, in json
ValueError: syntax error in JSON

4 comments

ricksorensen · 2023-11-22

This returns status_code==400 on my ESP32C3 , here is REPL :

>>> r = requests.get('https://aa.usno.navy.mil/api/rstt/oneday?date=2023-11-21 &coords=48.441832,9.890139 &tz=2')
>>> r.json()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "requests/__init__.py", line 33, in json
ValueError: syntax error in JSON
>>> r.text
'<html>\r\n<head><title>400 Bad Request</title></head>\r\n<body>\r\n<center><h1>400 Bad Request</h1></center>\r\n<hr><center>nginx</center>\r\n</body>\r\n</html>\r\n'
>>> r.status_code
400
>>>

The request works fine on my Linux box - using ipython- and returns a good json string. The text is not JSON. Maybe the difficulty is in requests module?

On the ESP32C3 micropython:

>>> r.headers
{'Server': 'nginx', 'Connection': 'close', 'Date': 'Wed, 22 Nov 2023 17:16:01 GMT', 'Content-Type': 'text/html; charset=utf-8', 'Content-Length': '150'}
>>>

On my linux box:

In [5]: r.headers
Out[5]: {'Server': 'nginx', 'Date': 'Wed, 22 Nov 2023 17:15:20 GMT', 'Content-Type': 'application/json', 'Content-Length': '1274', 'Connection': 'keep-alive', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains', 'X-Frame-Options': 'SAMEORIGIN', 'X-XSS-Protection': '1; mode=block', 'Content-Security-Policy': "default-src 'self';", 'Access-Control-Allow-Origin': '*'}

ricksorensen · 2023-11-22

If I remove the space between date and &coords and remove the space abetween coordinates and &tz it works:

>>> r=requests.get('https://aa.usno.navy.mil/api/rstt/oneday?date=2023-11-21&coords=48.441832,9.890139&tz=2')
>>> r.json()
{'apiversion': '4.0.1', 'properties': {'data': {'year': 2023, 'month': 11, 'fracillum': '61%', 'closestphase': {'phase': 'First Quarter', 'month': 11, 'year': 2023, 'time': '12:50', 'day': 20}, 'isdst': False, 'day': 21, 'day_of_week': 'Tuesday', 'label': None, 'sundata': [{'phen': 'Begin Civil Twilight', 'time': '08:02'}, {'phen': 'Rise', 'time': '08:37'}, {'phen': 'Upper Transit', 'time': '13:06'}, {'phen': 'Set', 'time': '17:35'}, {'phen': 'End Civil Twilight', 'time': '18:10'}], 'tz': 2.0, 'moondata': [{'phen': 'Set', 'time': '00:52'}, {'phen': 'Rise', 'time': '15:09'}, {'phen': 'Upper Transit', 'time': '20:34'}], 'curphase': 'Waxing Gibbous'}}, 'type': 'Feature', 'geometry': {'coordinates': [9.890139, 48.44183], 'type': 'Point'}}
>>>

permaBox · 2023-11-23

It works ... sometimes, means it is not stable.
I often get this error:
...
File "requests/init.py", line 180, in get
File "requests/init.py", line 76, in request
OSError: -202

jonnor · 2024-08-25

In order to be able to debug the issue, you need to provide examples of valid JSON that the library does not parse.

CANDIDATE · ISSUE

JSON module does not throw exception when object is not serialisable

openby peterhinchopened 2015-12-14updated 2025-07-15

bug

This came up in http://forum.micropython.org/viewtopic.php?f=6&t=953&p=7551#p7551
cPython throws an exception on an attempt to serialise a bytes object (regardless of object contents):

>>> a = bytes(x for x in range(256))
>>> z = json.dumps(a).encode('utf8')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.4/json/__init__.py", line 230, in dumps
    return _default_encoder.encode(obj)
  File "/usr/lib/python3.4/json/encoder.py", line 192, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/lib/python3.4/json/encoder.py", line 250, in iterencode
    return _iterencode(o, 0)
  File "/usr/lib/python3.4/json/encoder.py", line 173, in default
    raise TypeError(repr(o) + " is not JSON serializable")
TypeError: b'\x00\x01\x02\x03\x04\x05\x06\x07\x08\t\n\x0b\x0c\r\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f !"#$%&\'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\\]^_`abcdefghijklmnopqrstuvwxyz{|}~\x7f\x80\x81\x82\x83\x84\x85\x86\x87\x88\x89\x8a\x8b\x8c\x8d\x8e\x8f\x90\x91\x92\x93\x94\x95\x96\x97\x98\x99\x9a\x9b\x9c\x9d\x9e\x9f\xa0\xa1\xa2\xa3\xa4\xa5\xa6\xa7\xa8\xa9\xaa\xab\xac\xad\xae\xaf\xb0\xb1\xb2\xb3\xb4\xb5\xb6\xb7\xb8\xb9\xba\xbb\xbc\xbd\xbe\xbf\xc0\xc1\xc2\xc3\xc4\xc5\xc6\xc7\xc8\xc9\xca\xcb\xcc\xcd\xce\xcf\xd0\xd1\xd2\xd3\xd4\xd5\xd6\xd7\xd8\xd9\xda\xdb\xdc\xdd\xde\xdf\xe0\xe1\xe2\xe3\xe4\xe5\xe6\xe7\xe8\xe9\xea\xeb\xec\xed\xee\xef\xf0\xf1\xf2\xf3\xf4\xf5\xf6\xf7\xf8\xf9\xfa\xfb\xfc\xfd\xfe\xff' is not JSON serializable

MicroPython does not throw an exception but does not work: the restored object is of type str and its contents differs from that saved:

>>> a = bytes(x for x in range(256))
>>> z = json.dumps(a)
>>> x = json.loads(z)
>>> len(x)
192
>>> len(a)
256
>>> type(a)
<class 'bytes'>
>>> type(x)
<class 'str'>
>>>

json lib module, syntax error

JSON module does not throw exception when object is not serialisable

Keyboard