asyncio: MicroPython vs. CPython create_task() garbage collection behavior
(Not sure if I should label this a docs bug, feature request, or discussion. Apologies if I missed the mark.)
CPython exhibits a bit of a counterintuitive behavior, which requires users to store the result of asyncio.create_task(), otherwise face the danger of the task being garbage collected.
Will McGugan of Textual blogged about this here:
https://textual.textualize.io/blog/2023/02/11/the-heisenbug-lurking-in-your-async-code/
Vincent Bernat reported this as a documentation bug against CPython, https://github.com/python/cpython/issues/88831 last year, and subsequently the CPython documentation was adjusted to say:
Important
Save a reference to the result of this function, to avoid a task disappearing mid-execution. The event loop only keeps weak references to tasks. A task that isn’t referenced elsewhere may get garbage collected at any time, even before it’s done. For reliable “fire-and-forget” background tasks, gather them in a collection:
background_tasks = set()
for i in range(10):
task = asyncio.create_task(some_coro(param=i))
# Add task to the set. This creates a strong reference.
background_tasks.add(task)
# To prevent keeping references to finished tasks forever,
# make each task remove its own reference from the set after
# completion:
task.add_done_callback(background_tasks.discard)
Is MicroPython exhibiting the same behavior? If so, should the documentation be adjusted?
- The MicroPython documentation provides an example that does not store a reference, basically what CPython warns users not to do. Is this a misleading example or a non-issue in MicroPython? Should the example be CPython-friendly anyway though? Should the documentation explain this difference?
- Even if one wanted to be better-safe-than-sorry and write portable code, the CPython-recommended code above does not work under MicroPython, resulting in a
TypeError: unsupported type for __hash__: 'Task'. Should__hash__be added for Tasks? Or should the documentation mention some other way to do this?
Not a MicroPython bug per se, but also worth mentioning: @peterhinch's (excellent!) async tutorial also has examples where Tasks are not stored. At one point it's mentioned that "[t]he .create_task method returns the Task instance which may be saved for status checking or cancellation" (emphasis mine). May, or should?
extmod/asyncio: Add `Task` methods from CPython
This updates the way that Task is defined in the C module for asyncio which implements tasks & task queues. The goal is to bring asyncio in MicroPython slightly closer to CPython.
This adds the following methods to Task:
get_coro()- how CPython exposes the coroutine backing the task (CPython Docs)result()- a helper method from CPython for returning the successful response (CPython Docs)exception()- a helper method from CPython for returning the raised values (CPython Docs)cancelled()- helper for true / false if the task has been cancelled (CPython Docs)add_done_callback()- adds a callback to the task for completion or if already complete executes it (CPython docs)remove_done_callback()- removes a specific "done" callback from the task (CPython docs)
~Adds a hash unary operation so Tasks can be added to sets like in CPython.~
~Also adds set_result and set_exception for consistency but neither of these do anything - they raise a RuntimeError because they aren't supported on Tasks. :shrug:~