QUERY · ISSUE

Passing tests reported as failures with `MICROPY_ERROR_REPORTING_TERSE`

openby agattiopened 2025-07-18updated 2025-07-18

bug

Port, board and/or hardware

Any

MicroPython version

MicroPython v1.26.0-preview.386.g17fbc5abd.dirty

Reproduction

Clean build your port of choice
Run the test suite, notice the absence of failures
Add #define MICROPY_ERROR_REPORTING (MICROPY_ERROR_REPORTING_TERSE) to your port's mpconfigport.h file
Clean build the port
Run the test suite again.

Expected behaviour

All tests that were passing before changing the error reporting level should still pass.

Observed behaviour

I've seen the following tests fail (for example on QEMU/RV32):

micropython/heapalloc_exc_compressed.py
micropython/heapalloc_exc_compressed_emg_exc.py
micropython/native_with.py
micropython/opt_level_lineno.py
micropython/viper_with.py
misc/print_exception.py

Probably there are more of these but are port-specific and haven't encountered them yet.

Additional Information

The issue is that those tests perform an exact string match on the exception message being raised by the interpreter. With the error reporting level changing, the exception message may change, and thus fail perfectly working tests.

I see this issue on a custom port I'm doing where, for space reasons, I've got to keep error reporting verboseness down. This means I cannot get a single clean test run because with the extra space taken by the error messages (and their interpolation) I have to cut out other features to fit things in the allotted flash space.

A similar issue also exists, as in it's not possible to have a clean test run with the port's ROM level set to MICROPY_CONFIG_ROM_LEVEL_MINIMUM as some tests (even in basic) depend on features cut out by that definition.

I don't mind cleaning up those tests to have a more comprehensive pass/fail check and eventually migrate them to unittest if that's required (and putting in the work to make it testable with the minimum feature set, whilst I'm here...), but before starting this I wonder if this is a PR (or multiple PRs, who knows) that will be at least considered for inclusion.

Code of Conduct

Yes, I agree

CANDIDATE · ISSUE

esp32: viper tests fail after a while

openby tveopened 2020-04-11updated 2020-04-13

port-esp32

If I execute a micropython/viper test on an esp32 in a loop it starts failing after a while (I started with a HW reset before this):

> while true; ./run-tests --device /dev/ttyUSB0 --target esp32 micropython/
viper_ptr32_load.py; end
pass  micropython/viper_ptr32_load.py
1 tests performed (4 individual testcases)
1 tests passed
[...]
pass  micropython/viper_ptr32_load.py
1 tests performed (4 individual testcases)
1 tests passed
FAIL  micropython/viper_ptr32_load.py
1 tests performed (4 individual testcases)
0 tests passed
1 tests failed: viper_ptr32_load
skip  micropython/viper_ptr32_load.py
0 tests performed (0 individual testcases)
0 tests passed
1 tests skipped: viper_ptr32_load

It takes less than a minute to get there. I noticed because running the full test suite shows this, e.g.:

> ./run-tests --device /dev/ttyUSB0 --target esp32 -d micropython
[...]
pass  micropython/viper_misc_intbig.py
pass  micropython/viper_ptr16_load.py
pass  micropython/viper_ptr16_store.py
pass  micropython/viper_ptr32_load.py
pass  micropython/viper_ptr32_store.py
FAIL  micropython/viper_ptr8_load.py
FAIL  micropython/viper_ptr8_store.py
FAIL  micropython/viper_subscr.py
FAIL  micropython/viper_try.py
FAIL  micropython/viper_types.py
FAIL  micropython/viper_with.py
65 tests performed (411 individual testcases)
59 tests passed
4 tests skipped: heap_locked heapalloc_bytesio2 meminfo memstats
6 tests failed: viper_ptr8_load viper_ptr8_store viper_subscr viper_try viper_types viper_with

The tests that fail at the end are because they're at the end, not because they fail per-se.