QUERY · ISSUE

Passing tests reported as failures with `MICROPY_ERROR_REPORTING_TERSE`

openby agattiopened 2025-07-18updated 2025-07-18

bug

Port, board and/or hardware

Any

MicroPython version

MicroPython v1.26.0-preview.386.g17fbc5abd.dirty

Reproduction

Clean build your port of choice
Run the test suite, notice the absence of failures
Add #define MICROPY_ERROR_REPORTING (MICROPY_ERROR_REPORTING_TERSE) to your port's mpconfigport.h file
Clean build the port
Run the test suite again.

Expected behaviour

All tests that were passing before changing the error reporting level should still pass.

Observed behaviour

I've seen the following tests fail (for example on QEMU/RV32):

micropython/heapalloc_exc_compressed.py
micropython/heapalloc_exc_compressed_emg_exc.py
micropython/native_with.py
micropython/opt_level_lineno.py
micropython/viper_with.py
misc/print_exception.py

Probably there are more of these but are port-specific and haven't encountered them yet.

Additional Information

The issue is that those tests perform an exact string match on the exception message being raised by the interpreter. With the error reporting level changing, the exception message may change, and thus fail perfectly working tests.

I see this issue on a custom port I'm doing where, for space reasons, I've got to keep error reporting verboseness down. This means I cannot get a single clean test run because with the extra space taken by the error messages (and their interpolation) I have to cut out other features to fit things in the allotted flash space.

A similar issue also exists, as in it's not possible to have a clean test run with the port's ROM level set to MICROPY_CONFIG_ROM_LEVEL_MINIMUM as some tests (even in basic) depend on features cut out by that definition.

I don't mind cleaning up those tests to have a more comprehensive pass/fail check and eventually migrate them to unittest if that's required (and putting in the work to make it testable with the minimum feature set, whilst I'm here...), but before starting this I wonder if this is a PR (or multiple PRs, who knows) that will be at least considered for inclusion.

Code of Conduct

Yes, I agree

CANDIDATE · PULL REQUEST

tests/micropython: Make tests behave in low memory condition.

mergedby agattiopened 2025-08-19updated 2025-08-26

tests

Summary

This PR changes the "viper_ptr*_store_boundary" tests to make them fail more gracefully in low memory conditions.

The original version of the tests compiled viper code blocks on the fly when it needed them, making them fail at runtime on some boards that do not come with enough memory for this test. This clashes with "run-tests.py"'s ability to look for a particular signature to mark tests as skipped due to not enough memory.

Now compiled code blocks are generated at the beginning of the test inside an appropriate exception handler. In case of a memory error when pre-compiling a code block, the running test exits reporting a low memory condition to the test runner. This allows to have clean test runs on all platforms when it comes to viper pointer tests.

Testing

Modified tests were run on an ESP8266 without any parameters to make sure they were reported as too large, and then with an appropriate prologue file to make sure the output is correct. The same tests were also executed on the Unix port on x64, just to be sure.

Trade-offs and Alternatives

I've tried to simplify certain parts of the test to counter the added complexity of generating blocks upfront, although it came with a test output data rearrangement to make it work. On the other hand, now those tests share most of their code except for the typed getters and setters.