SEGV in decompress_error_text_maybe
Description
We observed MicroPython accessed a bogus address in decompress_error_text_maybe while referencing an Error object from builtins. One more interesting detail is the PoCs are all using sys.version to initialize Error. We have attached three PoCs below.
Proof of Concept
$ # build unix port with ASAN, at the root source code directory.
$ export CC=clang
$ export CXX=clang++
$ export CFLAGS="-fsanitize=address -fno-omit-frame-pointer"
$ export CXXFLAGS=$CFLAGS
$ export LDFLAGS=$CFLAGS
$ export DEBUG=1
$ make -C mpy-cross -j
$ make -C ports/unix -j all lib
$
$ # run a poc.
$ export ASAN_OPTIONS="detect_leaks=0"
$ ./ports/unix/build-standard/micropython <poc_file>
Environment
Ubuntu 20.04
Intel(R) Xeon(R) Gold 5218 CPU @ 2.30GHz
Memory: 64 GB
Affected Version
v1.20.0 (commit a3862e726, latest as of 2023-09-26)
v1.20.0 (commit 813d559bc, 2023-06-19)
Discovered in the UNIX port version.
Uninitialized memory read in the lexer part
Description
We observed a compatibility issue with MemorySanitizer. When compiled with MemorySanitizer, the UNIX port of micropython crashed at startup.
Proof of Concept
$ # build unix port with ASAN, at the root source code directory.
$ export CC=clang
$ export CXX=clang++
$ export CFLAGS="-fsanitize=memory -fno-omit-frame-pointer"
$ export CXXFLAGS=$CFLAGS
$ export LDFLAGS=$CFLAGS
$ export DEBUG=1
$ make -C mpy-cross -j
$ make -C ports/unix -j all lib
$
$ # run a poc.
$ export ASAN_OPTIONS="detect_leaks=0"
$ ./ports/unix/build-standard/micropython
Please notice that this crash happened without any argument to micropython, as long as it was compiled with memory sanitizer.
Issue
In https://github.com/micropython/micropython/blob/master/py/lexer.c#L153, lex->fstring_args_idx was referenced before being initialized, causing a crash when compiled with memory sanitizer.
if (lex->fstring_args_idx) {
// if there are saved chars, then we're currently injecting fstring args
if (lex->fstring_args_idx < lex->fstring_args.len) {
lex->chr2 = lex->fstring_args.buf[lex->fstring_args_idx++];
} else {
// no more fstring arg bytes
lex->chr2 = '\0';
}
}
Fix
Use memset to initialize the lex struct in https://github.com/micropython/micropython/blob/master/py/lexer.c#L837 .
memset(lex, 0, sizeof(*lex)); // insert this line
lex->reader = reader;
lex->line = 1;
lex->column = (size_t)-2; // account for 3 dummy bytes
lex->emit_dent = 0;
lex->nested_bracket_level = 0;
lex->alloc_indent_level = MICROPY_ALLOC_LEXER_INDENT_INIT;
lex->num_indent_level = 1;
lex->indent_level = m_new(uint16_t, lex->alloc_indent_level);
//...
Environment
Ubuntu 20.04
Intel(R) Xeon(R) Gold 5218 CPU @ 2.30GHz
Memory: 64 GB
Affected Version
v1.20.0 (commit a3862e726, latest as of 2023-09-26)
v1.20.0 (commit 813d559bc, 2023-06-19)
Comment
The issue analysis and the possible fix were done and written by @chibinz.