TLS errors on ESP-IDF 5.4 with default MICROPY_GC_INITIAL_HEAP_SIZE
Port, board and/or hardware
esp32 port
MicroPython version
➜ esp32 git:(master) git describe --dirty
v1.24.0-224-ga4ab84768
Reproduction
Making single requests to 2 different TLS endpoints (which have small payloads).
➜ esp32 git:(master) ✗ idf.py --version
ESP-IDF v5.4
...
Chip is ESP32-D0WDQ6 (revision v0.0)
Features: WiFi, BT, Dual Core, Coding Scheme None
Crystal is 40MHz
Expected behaviour
IDF 5.3 was stable without change to MICROPY_GC_INITIAL_HEAP_SIZE
➜ esp32 git:(master) idf.py --version
ESP-IDF v5.3
Observed behaviour
I was getting various intermittent lock up (including no ping responses via network to device) or OSError with TLS HTTP requests:
(-17040, 'MBEDTLS_ERR_RSA_PUBLIC_FAILED+MBEDTLS_ERR_MPI_ALLOC_FAILED')
(-29312, 'MBEDTLS_ERR_SSL_CONN_EOF')
[Errno 104] ECONNRESET
Workaround: Adjusting MICROPY_GC_INITIAL_HEAP_SIZE to (52 * 1024) (from 56) none of these errors triggered. Errors triggered if I had the value set to (54 * 1024) as well.
Additional Information
It is unstable on 5.3.2 as well, but I noted it is not in the "approved" IDF versions. I am not sure if the patch version matters, but because 5.2 and 5.2.2 are explicitly listed, it makes me think they are. Unfortunately, IDF does not have a 5.3.0, so I think micropython esp32 port just lists 5.3. May be worth indicating that supported patch versions will be explicitly listed.
it was relatively stable on 5.3.1 (not as bad as 5.3.2)
Code of Conduct
Yes, I agree
ESP32: Micropython allocated heap is reduced for IDF >= 4.2.2.
Micropython allocates a heap size of only 65536 bytes on the ESP32 for later versions of the ESP IDF (>= 4.2.2) instead of the actually available 110536 bytes.
Since the implementationof a new allocator in the IDF (October 2020), heap_cap_get_largest_block() returns the size of the largest allocatable block rounded down to the nearest power of 2 (in contradiction with the documentation.
I have filed an issue against the IDF at https://github.com/espressif/esp-idf/issues/7808.
The follwing patch provides an ugly work around:
diff --git a/ports/esp32/main.c b/ports/esp32/main.c
index 2ba613668..dce82b7fb 100644
--- a/ports/esp32/main.c
+++ b/ports/esp32/main.c
@@ -131,7 +131,20 @@ void mp_task(void *pvParameter) {
#else
// Allocate the uPy heap using malloc and get the largest available region
size_t mp_task_heap_size = heap_caps_get_largest_free_block(MALLOC_CAP_8BIT);
- void *mp_task_heap = malloc(mp_task_heap_size);
+ void *mp_task_heap;
+ for (size_t incr = mp_task_heap_size >> 1;
+ incr > 4;
+ incr >>= 1)
+ {
+ mp_task_heap_size += incr;
+ mp_task_heap = heap_caps_malloc(mp_task_heap_size, MALLOC_CAP_8BIT);
+ if (mp_task_heap == NULL) {
+ mp_task_heap_size -= incr;
+ }
+ heap_caps_free(mp_task_heap);
+ printf("HeapSize=0x%x %d\n", mp_task_heap_size, mp_task_heap_size);
+ }
+ mp_task_heap = malloc(mp_task_heap_size);
#endif
soft_reset: