Why PC .c files full path strings are in built Micropython firmware (usermod) ?
Port, board and/or hardware
esp32
MicroPython version
1.26.1
Reproduction
.
Expected behaviour
.
Observed behaviour
I'm upgrading from MPY 1.25 + IDF 5.3 to MPY 1.26.1 + IDF 5.4.2 , for ESP32C3. I built my MPY firmware. Using a lot of custom code in usermod.
Using strings micropython.bin command, I found that newly generated micropython.bin contains many usermod .c file absolute PC path strings:
/onpc/long/path/micropython/ports/esp32/usermod/aa.c
/onpc/long/path/micropython/ports/esp32/usermod/xxxx/gg.c
....
(only usermod .c file paths appeared. Other MPY .c file path not)
Although those PC path strings won't take 200KB, is their being there in firmware is a config bug? (Since MPY 1.26 or since IDF 5.4 ?)
I checked firmware files I built before upgrading (also have usermod). Those path strings were not in old firmwares.
When you have many .c files in usermod, those strings can take tens of KB, which is not good for small-Flash mcu.
Additional Information
I didn't add any debugging or logging flag to build config.
My .cmake in usermod has always be like:
add_library(usermod_xxxx INTERFACE)
target_sources(usermod_xxxx INTERFACE
${CMAKE_CURRENT_LIST_DIR}/xxxx.c
${CMAKE_CURRENT_LIST_DIR}/xxxxx.c
.....
)
target_include_directories(usermod_xxxx INTERFACE
${CMAKE_CURRENT_LIST_DIR}
)
target_link_libraries(usermod INTERFACE usermod_xxxx)
Code of Conduct
Yes, I agree
esp32: register user C modules with IDF component manager and handle relative usermod paths
Summary
This change adds the possibility to define usermod specific IDF manifests and improves how user-provided C modules with relative paths are handled when building the ESP32 port:
- Register user module directories with the ESP-IDF Component Manager when the user module directory contains both idf_component.yml and CMakeLists.txt and append those directories to EXTRA_COMPONENT_DIRS so the IDF can locate them correctly.
- Convert configured USER_C_MODULES entries to absolute paths and replace the original variable with those absolute paths for later CMake stages. This makes relative module paths safe and reliable across the build.
Testing
Built MicroPython locally for BOARD=ESP32_GENERIC_S3 and BOARD_VARIANT=SPIRAM_OCT with a user module passed as absolute and relative path. The build contained the needed files.
The build completed successfully.
Trade-offs and Alternatives
The code size of the main CMakeLists.txt is 20 lines longer. On the other hand, you don't need to touch the micropython code to build with custom user modules with ESP components integrated.
Also, providing relative paths to the build changes behavior. This should be documented. On the other hand, this will prevent such issues like https://github.com/micropython/micropython/issues/14352.
(Maintainer edit: Closes #14352 )