asyncio: MicroPython vs. CPython create_task() garbage collection behavior
(Not sure if I should label this a docs bug, feature request, or discussion. Apologies if I missed the mark.)
CPython exhibits a bit of a counterintuitive behavior, which requires users to store the result of asyncio.create_task(), otherwise face the danger of the task being garbage collected.
Will McGugan of Textual blogged about this here:
https://textual.textualize.io/blog/2023/02/11/the-heisenbug-lurking-in-your-async-code/
Vincent Bernat reported this as a documentation bug against CPython, https://github.com/python/cpython/issues/88831 last year, and subsequently the CPython documentation was adjusted to say:
Important
Save a reference to the result of this function, to avoid a task disappearing mid-execution. The event loop only keeps weak references to tasks. A task that isn’t referenced elsewhere may get garbage collected at any time, even before it’s done. For reliable “fire-and-forget” background tasks, gather them in a collection:
background_tasks = set()
for i in range(10):
task = asyncio.create_task(some_coro(param=i))
# Add task to the set. This creates a strong reference.
background_tasks.add(task)
# To prevent keeping references to finished tasks forever,
# make each task remove its own reference from the set after
# completion:
task.add_done_callback(background_tasks.discard)
Is MicroPython exhibiting the same behavior? If so, should the documentation be adjusted?
- The MicroPython documentation provides an example that does not store a reference, basically what CPython warns users not to do. Is this a misleading example or a non-issue in MicroPython? Should the example be CPython-friendly anyway though? Should the documentation explain this difference?
- Even if one wanted to be better-safe-than-sorry and write portable code, the CPython-recommended code above does not work under MicroPython, resulting in a
TypeError: unsupported type for __hash__: 'Task'. Should__hash__be added for Tasks? Or should the documentation mention some other way to do this?
Not a MicroPython bug per se, but also worth mentioning: @peterhinch's (excellent!) async tutorial also has examples where Tasks are not stored. At one point it's mentioned that "[t]he .create_task method returns the Task instance which may be saved for status checking or cancellation" (emphasis mine). May, or should?
Incorrect uasyncio behaviour: tasks are not correctly terminated
(First reported on the forum)
The following snippet:
try:
import uasyncio as asyncio
except ImportError:
import asyncio
async def count():
i = 0
while True:
print(i)
i += 1
await asyncio.sleep(1)
async def main():
asyncio.create_task(count())
await asyncio.sleep(5)
asyncio.run(main())
asyncio.run(main())
Produces the following in CPython:
0
1
2
3
4
0
1
2
3
4
But in MicroPython:
0
1
2
3
4
0
5
1
6
2
7
3
8
4
9
It appears that tasks are not correctly terminated (?).
A workaround that produces the correct output is to run the second task in a second event loop:
try:
import uasyncio as asyncio
except ImportError:
import asyncio
async def count():
i = 0
while True:
print(i)
i += 1
await asyncio.sleep(1)
async def main():
asyncio.create_task(count())
await asyncio.sleep(5)
asyncio.run(main())
asyncio.new_event_loop() # Added line
asyncio.run(main())
Note that running either of the snippets repeatedly causes more confusing output because all previous tasks are still running on the queue.