Incorrect static const data behavior in native module on esp32.
Checks
-
I agree to follow the MicroPython Code of Conduct to ensure a safe and respectful space for everyone.
-
I've searched for existing issues matching this bug, and didn't find any.
Port, board and/or hardware
esp32
MicroPython version
MicroPython v1.22.2 on 2024-02-22; Generic ESP32 module with ESP32
Reproduction
I have a medium-complex native module that performs lossless data compression/decompression.
My code as a native modules has been tested to work on x64 and armv6m. I'm currently testing on esp32 and my code is not longer working; I have narrowed it down to the following:
I have a few static const variables defined here. If I remove the static, the code works fine. However, I tried adding static to examples/natmod/features1, and that code works fine with a static. So I have no idea what's going on here.
I found this out because write_to_bit_buffer was being invoked with n_bits=32 at one point, and I traced this down to huffman_bits[0] == 32, but obviously the value should be the constant 0x2.
Expected behaviour
If my library is operating correctly, it should have the following behavior:
>>> import tamp
>>> tamp.compress(b"foo")
b'X\xb3\x04\x18'
>>> tamp.decompress(tamp.compress(b"foo"))
bytearray(b'foo')
Observed behaviour
Accessing the RO data results in incorrect values.
Additional Information
No, I've provided everything above.
Unsigned conversion overflow for QSTRs during build
Checks
-
I agree to follow the MicroPython Code of Conduct to ensure a safe and respectful space for everyone.
-
I've searched for existing issues matching this bug, and didn't find any.
Port, board and/or hardware
Applies to all ports and boards
MicroPython version
Micropython v1.22
Reproduction
- Add a new ext_mod with lots of QSTRs
Expected behaviour
Build succeeds
Observed behaviour
Build creates the following error messages
micropython-v1.22.2/py/scope.c:36:22: warning: unsigned conversion from 'int' to 'unsigned char' changes value from '256' to '0' [-Woverflow]
36 | [SCOPE_LAMBDA] = MP_QSTR__lt_lambda_gt_,
micropython-v1.22.2/py/scope.c:46:5: note: in expansion of macro 'MP_STATIC_ASSERT'
46 | MP_STATIC_ASSERT(MP_QSTR__lt_lambda_gt_ <= UINT8_MAX);
| ^~~~~~~~~~~~~~~~
and a few similar errors.
Additional Information
In py/scope.c, a comment says that "these low numbered qstrs should fit in 8 bits. See assertions below." However, during the QSTR generation, there is nothing to force this.
It seems that there is a way to do this in py/makeqstrdefs.py: there is a list of operators operator_qstr_list that must have low values.
My solution was to add these values to this list.
"<module>",
"<lambda>",
"<listcomp>",
"<dictcomp>",
"<setcomp>",
"<genexpr>",
However, the name of the list and the comment above it are now incorrect, as they refer to 'operators'.
Please indicate whether it would be better to create another list with additional QSTRs to keep below 255, and add another list to the test around line 410 in print_qstr_data, changing
pool = 0 if qstr in operator_qstr_list else 1
to something like
pool = 0 is qstr in operator_qstr_list or qstr in more_low_qstr_list else 1