Uasyncio cancelling coroutine takes as long as next scheduled execution
Since I didn't see any open issues about it and jimmo asked, I though I'd open this long-standing issue:
If you cancel a coroutine, the exception won't be raised before the coroutine is scheduled next. This is because it just stays in waitq until it would be executed regularly.
So a coroutine yielding an await uasyncio.sleep(10) will only receive the cancel exception after those 10 seconds have passed.
This results in very late exception handling inside the coroutine but also in canceled coroutines being "stuck" in waitq so the user code has to be careful not to cause a queue overflow if he uses cancellations often.
The fix however requires more than a change to uasyncio, it requires changes to utimeq too so a coroutine can be removed from the queue.
As I understand it Paul has that fixed on his fork.
@peterhinch implemented a high level workaround on his fast_io implementation for making coroutines cancel quickly by using an additional queue but even with that implementation coroutines stay in the utimeq until they are scheduled regularly.
extmod/uasyncio: wait_for cancellation bugs
CancelledError forwarding if wait_for got cancelled (not the task it awaits):
Your assumption "# Ignore CancelledError from aw, it's probably due to timeout" in funcy.py line 20 https://github.com/micropython/micropython/blob/ad004db662b383a9bf310601e13e896c99e11e30/extmod/uasyncio/funcs.py#L20 is incorrect.
As you can see in the testcase "wait_for.test_cancellation_forwarded", if you cancel the caller task of "wait_for", the coro awaited by "wait_for" gets correctly cancelled, but the caller of "wait_for" never gets the Exception raised. The CancelledError gets swallowed by "wait_for" instead of forwarding it.
Wait_for also behaves incorrectly if CancelldError Exceptions are caught inside the task it awaits because they don't get raised to wait_for anymore. However Cpython does also raise those to wait_for even if they are caught in the task wait_for awaits.
All my wait_for testcases (working on upy and cpy) are here: https://gist.github.com/kevinkk525/3313a8d2763031a705e37d1748d1d996
The one in question for all bugs mentioned:
async def test_cancellation_forwarded(catch=False, catch_inside=False):
res = [False, False]
async def wait():
nonlocal res
try:
await asyncio.wait_for(awaiting(2, catch_inside), 0.5)
except asyncio.TimeoutError:
dprint("Got timeout error")
res[0] = asyncio.TimeoutError
raise
except asyncio.CancelledError:
dprint("Got canceled")
if catch:
res[0] = True
else:
raise
async def killer(t):
dprint("killer started")
await asyncio.sleep(0.2)
dprint("killing wait()")
t.cancel()
t = asyncio.create_task(wait())
k = asyncio.create_task(killer(t))
try:
await t
except asyncio.CancelledError:
dprint("waiting got cancelled")
res[1] = asyncio.CancelledError
return res