esp32: viper tests fail after a while
If I execute a micropython/viper test on an esp32 in a loop it starts failing after a while (I started with a HW reset before this):
> while true; ./run-tests --device /dev/ttyUSB0 --target esp32 micropython/
viper_ptr32_load.py; end
pass micropython/viper_ptr32_load.py
1 tests performed (4 individual testcases)
1 tests passed
[...]
pass micropython/viper_ptr32_load.py
1 tests performed (4 individual testcases)
1 tests passed
FAIL micropython/viper_ptr32_load.py
1 tests performed (4 individual testcases)
0 tests passed
1 tests failed: viper_ptr32_load
skip micropython/viper_ptr32_load.py
0 tests performed (0 individual testcases)
0 tests passed
1 tests skipped: viper_ptr32_load
It takes less than a minute to get there. I noticed because running the full test suite shows this, e.g.:
> ./run-tests --device /dev/ttyUSB0 --target esp32 -d micropython
[...]
pass micropython/viper_misc_intbig.py
pass micropython/viper_ptr16_load.py
pass micropython/viper_ptr16_store.py
pass micropython/viper_ptr32_load.py
pass micropython/viper_ptr32_store.py
FAIL micropython/viper_ptr8_load.py
FAIL micropython/viper_ptr8_store.py
FAIL micropython/viper_subscr.py
FAIL micropython/viper_try.py
FAIL micropython/viper_types.py
FAIL micropython/viper_with.py
65 tests performed (411 individual testcases)
59 tests passed
4 tests skipped: heap_locked heapalloc_bytesio2 meminfo memstats
6 tests failed: viper_ptr8_load viper_ptr8_store viper_subscr viper_try viper_types viper_with
The tests that fail at the end are because they're at the end, not because they fail per-se.
Need way to check for availability of micropython.native/micropython.viper
As was mentioned previously, we currently have testsuite failures on platforms which don't have micropython.native/micropython.viper - due to tests which use those unconditionally. See #856 for example.
I tried to look into it, and figured there's no clean way to do that: SyntaxError for them is handled in adhoc manner, not via NLR, so it's not possible to catch. But using NLR wouldn't help either, because that handling happens on compilation phase, not during execution, so user code cannot catch it in any way.
So, the only way I see to deal with the issue (and the issue is broken testsuite) is to have adhoc support for micropython.native/micropython.viper in run-tests (i.e. it will first detect if they're available, and only then will run tests which are known to use it, as individual scripts cannot detect it on their own level).
Any better ideas?