Crashes in mp_pairheap_delete
Description
The UNIX port of MicroPython crashes in the programs involving asyncio.Event(). I briefly looked upon a similar issue and issue #6868 seems like pointing to a similar crash (found with a static analyzer in 2021). This crash is also exhibited in either global-buffer-overflow or null-dereference. I have attached three PoCs for each bug type below.
Proof of Concept
$ # build unix port with ASAN, at the root source code directory.
$ export CC=clang
$ export CXX=clang++
$ export CFLAGS="-fsanitize=address -fno-omit-frame-pointer"
$ export CXXFLAGS=$CFLAGS
$ export LDFLAGS=$CFLAGS
$ export DEBUG=1
$ make -C mpy-cross -j
$ make -C ports/unix -j all lib
$
$ # run a poc.
$ export ASAN_OPTIONS="detect_leaks=0"
$ ./ports/unix/build-standard/micropython <poc_file>
Environment
Ubuntu 20.04
Intel(R) Xeon(R) Gold 5218 CPU @ 2.30GHz
Memory: 64 GB
Affected Version
v1.20.0 (commit a3862e726, latest as of 2023-09-26)
v1.20.0 (commit 813d559bc, 2023-06-19)
Discovered in the UNIX port version.
Uninitialized memory read in the lexer part
Description
We observed a compatibility issue with MemorySanitizer. When compiled with MemorySanitizer, the UNIX port of micropython crashed at startup.
Proof of Concept
$ # build unix port with ASAN, at the root source code directory.
$ export CC=clang
$ export CXX=clang++
$ export CFLAGS="-fsanitize=memory -fno-omit-frame-pointer"
$ export CXXFLAGS=$CFLAGS
$ export LDFLAGS=$CFLAGS
$ export DEBUG=1
$ make -C mpy-cross -j
$ make -C ports/unix -j all lib
$
$ # run a poc.
$ export ASAN_OPTIONS="detect_leaks=0"
$ ./ports/unix/build-standard/micropython
Please notice that this crash happened without any argument to micropython, as long as it was compiled with memory sanitizer.
Issue
In https://github.com/micropython/micropython/blob/master/py/lexer.c#L153, lex->fstring_args_idx was referenced before being initialized, causing a crash when compiled with memory sanitizer.
if (lex->fstring_args_idx) {
// if there are saved chars, then we're currently injecting fstring args
if (lex->fstring_args_idx < lex->fstring_args.len) {
lex->chr2 = lex->fstring_args.buf[lex->fstring_args_idx++];
} else {
// no more fstring arg bytes
lex->chr2 = '\0';
}
}
Fix
Use memset to initialize the lex struct in https://github.com/micropython/micropython/blob/master/py/lexer.c#L837 .
memset(lex, 0, sizeof(*lex)); // insert this line
lex->reader = reader;
lex->line = 1;
lex->column = (size_t)-2; // account for 3 dummy bytes
lex->emit_dent = 0;
lex->nested_bracket_level = 0;
lex->alloc_indent_level = MICROPY_ALLOC_LEXER_INDENT_INIT;
lex->num_indent_level = 1;
lex->indent_level = m_new(uint16_t, lex->alloc_indent_level);
//...
Environment
Ubuntu 20.04
Intel(R) Xeon(R) Gold 5218 CPU @ 2.30GHz
Memory: 64 GB
Affected Version
v1.20.0 (commit a3862e726, latest as of 2023-09-26)
v1.20.0 (commit 813d559bc, 2023-06-19)
Comment
The issue analysis and the possible fix were done and written by @chibinz.