esp32/mpconfigport: MICROPY_EVENT_POLL_HOOK fails to yield
On the STM32 MICROPY_EVENT_POLL_HOOK contains either a pyb_thread_yield() or a __WFI() causing the flow of execution to pause for a tad and thereby give other threads a chance at the cpu or provide an opportunity to save power. On the esp32 MICROPY_EVENT_POLL_HOOK does not contain any form of yield (https://github.com/micropython/micropython/blob/master/ports/esp32/mpconfigport.h#L245-L252). Instead, mp_hal_delay_ms both calls MICROPY_EVENT_POLL_HOOK and yields (https://github.com/micropython/micropython/blob/master/ports/esp32/mphalport.c#L164-L165) and mp_hal_stdin_rx_chr also calls both. However, in modselect poll_poll_internal just has the MICROPY_EVENT_POLL_HOOK macro (https://github.com/micropython/micropython/blob/master/extmod/moduselect.c#L256) with the result that on the esp32 it busy-waits, hogging the cpu, while on stm32 it yields as expected.
It's pretty easy to fix the esp32 situation and that's actually something included in #5473 (https://github.com/micropython/micropython/pull/5473/files#diff-b9499fc8ad5b9793822626f6befdb1b6 and https://github.com/micropython/micropython/pull/5473/files#diff-4c3d68ff23336da03f336fbc26571f7b). But that's not the end of the story...
Due to the esp32's 100hz clock tick moving the yield into MICROPY_EVENT_POLL_HOOK results in a 10ms granularity when a routine uses MICROPY_EVENT_POLL_HOOK. extmod/uasyncio_basic.py works around this by including code that checks "is the remaining sleep duration shorter than a 10ms clock tick? if yes, then busy-wait". However, poll_poll_internal doesn't (and really shouldn't since it's not port-specific). The next thing that happens is that in the test suite extmod/uasyncio_basic.py verifies that sleeping in uasyncio for 20ms and 40ms has the desired effect and it turns out it doesn't on the esp32, rather, the sleep times are often 30ms and 50ms. This over-sleeping happens because asyncio's IOQueue uses ipoll to sleep, which now gets 10ms long yields inserted and and mentioned above doesn't compensate for that. So long story short, fixing the not-yielding issue causes over-sleeping when using uasyncio.
IMHO the best fix is to increase the clock tick rate on the esp32 to 1kHz, which would also help reduce latency when waiting to receive a packet but I'm happy to learn about other alternatives!
[ESP32] utime.sleep_ms does not free GIL on small intervals
When using multithreading on the ESP32 port, and using sleep_ms(1) to yield to another thread that may be waiting on the GIL, the sleep_ms implementation does not free the GIL when milliseconds is approximately less than 10ms. This is because there is an early break from the sleep loop that doesn't call MICROPY_EVENT_POLL_HOOK
https://github.com/micropython/micropython/blob/master/ports/esp32/mphalport.c#L120-L138
By changing the inner loop from
for (;;) {
uint64_t t1 = esp_timer_get_time();
dt = t1 - t0;
if (dt + portTICK_PERIOD_MS * 1000 >= us) {
// doing a vTaskDelay would take us beyond requested delay time
break;
}
MICROPY_EVENT_POLL_HOOK
ulTaskNotifyTake(pdFALSE, 1);
}
to
for (;;) {
MICROPY_EVENT_POLL_HOOK
uint64_t t1 = esp_timer_get_time();
dt = t1 - t0;
if (dt + portTICK_PERIOD_MS * 1000 >= us) {
// doing a vTaskDelay would take us beyond requested delay time
break;
}
ulTaskNotifyTake(pdFALSE, 1);
}
The GIL will always be freed from this thread, even if the sleep_ms duration is very small, allowing another thread to access python objects and perform work that the main thread may be waiting on.