TLS errors on ESP-IDF 5.4 with default MICROPY_GC_INITIAL_HEAP_SIZE
Port, board and/or hardware
esp32 port
MicroPython version
➜ esp32 git:(master) git describe --dirty
v1.24.0-224-ga4ab84768
Reproduction
Making single requests to 2 different TLS endpoints (which have small payloads).
➜ esp32 git:(master) ✗ idf.py --version
ESP-IDF v5.4
...
Chip is ESP32-D0WDQ6 (revision v0.0)
Features: WiFi, BT, Dual Core, Coding Scheme None
Crystal is 40MHz
Expected behaviour
IDF 5.3 was stable without change to MICROPY_GC_INITIAL_HEAP_SIZE
➜ esp32 git:(master) idf.py --version
ESP-IDF v5.3
Observed behaviour
I was getting various intermittent lock up (including no ping responses via network to device) or OSError with TLS HTTP requests:
(-17040, 'MBEDTLS_ERR_RSA_PUBLIC_FAILED+MBEDTLS_ERR_MPI_ALLOC_FAILED')
(-29312, 'MBEDTLS_ERR_SSL_CONN_EOF')
[Errno 104] ECONNRESET
Workaround: Adjusting MICROPY_GC_INITIAL_HEAP_SIZE to (52 * 1024) (from 56) none of these errors triggered. Errors triggered if I had the value set to (54 * 1024) as well.
Additional Information
It is unstable on 5.3.2 as well, but I noted it is not in the "approved" IDF versions. I am not sure if the patch version matters, but because 5.2 and 5.2.2 are explicitly listed, it makes me think they are. Unfortunately, IDF does not have a 5.3.0, so I think micropython esp32 port just lists 5.3. May be worth indicating that supported patch versions will be explicitly listed.
it was relatively stable on 5.3.1 (not as bad as 5.3.2)
Code of Conduct
Yes, I agree
esp32: Add MICROPY_GC_INITIAL_HEAP_SIZE option and tune it.
This PR consists of two commits.
First, there are changes to the GC to improve the calculation of the size of the next heap area when automatically expanding the heap:
- Compute the existing total size by counting the total number of GC blocks, and then using that to compute the corresponding number of bytes.
- Round the bytes value up to the nearest multiple of BYTES_PER_BLOCK.
This makes the calculation slightly simpler and more accurate, and makes sure that, in the case of growing from one area to two areas, the number of bytes allocated from the system for the second area is the same as the first. For example on esp32 with an initial area size of 65536 bytes, the subsequent allocation is also 65536 bytes. Previously it was a number that was not even a multiple of 2.
Second, a change to esp32 to add MICROPY_GC_INITIAL_HEAP_SIZE and tune its value. This gets back the old heap-size behaviour on ESP32, before auto-split-heap was introduced: after the heap is grown one time the size is 111936 bytes, with about 40k left for the IDF. That's enough to start WiFi and do a HTTPS request.