ESP8266/ESP32: inplace firmware upgrade using the image file stored on the file-system
In practice, it would be very useful to be able to upgrade firmware in place using the firmware image file on the file-system. In fact, all mobile devices (like smartphones), IoT devices (like routers, robot cleaners, smart fans and smart lights), can do firmware upgrade from the firmware image file that is downloaded from the cloud and stored on the file-system.
Thanks to @yaota8266, that ESP8266 MicroPython does have the OTA build, however, it is not very useful but very restrictive. In particular, it requires you:
- to press some button to start upgrade. So if the device is installed on the ceiling fan, ceiling light, or inside some big box appliance, it is very inconvenient to start firmware upgrade
- to set Wifi credentials into his system which should be handled by the software web server part, instead of the OTA module (quite a lot of redundancy)
- Moreover, it has to pre-allocate 2 big flash spaces for the firmware, where only 1 is active at a time. This wastes too much executable flash space (only the 1st 1MB). As MicroPython grows bigger with more functionalities, 1MB cannot fit 2 copies very soon.
Therefore, the most useful way of doing OTA is to do firmware upgrade from the internal flash storage file-system, triggered by calling a function or upon system reboot.
Super enhancement for ESP8266: software-initiated MicroPython base firmware update from image file on LFS2 file-system and dynamic frozen modules
This PR added support for software-initiated OTA/non-OTA firmware update from a firmware image file stored on the internal LFS2 file-system, and managing dynamic frozen modules.
- Full device firmware update (DFU) (from a file)
This provides a more useful and practical way of full MicroPython base firmware update. Typically, one can create an ESP8266 web server (using e.g., microWebSrv, microdot, etc.) and upload the new firmware image file firmware.bin onto the LFS2 file-system; after that, instruct the backend server to call inisetup.fwupdate('firmware.bin') to update to the new MicroPython firmware.
This implementation of DFU works by first reading through the entire firmware image file so as to dig out all sectors and offsets using a hook in LFS2. After that, invoke the IRAM function esp.DFU() to erase needed sectors, read the firmware image and write it to the flash sector by sector. This function and all the functions it calls must not lie in the ROM (that is mapped to the 1st 1MB of flash) so that the entire 1st 1MB can be flashed without the need to ping-ponging between 2 usable sections of the firmware image (as in yaota8266's OTA solution). Thus, this resolution allows MicroPython base code size to grow to the full 1MB (including the bootloader). Moreover, you can also use this DFU function to flash a non-MicroPython ESP8266 firmware image to switch into non-MicroPython ESP8266 firmware.
Usage:
from inisetup import fwupdate
fwupdate('firmware-image-file.bin', erase_all=False, safe_check=True, verbose=True)
By default, the file-system will not be touched unless erase_all=True is set.
- Dynamic frozen modules (DFM)
You can now add/remove/list dynamic frozen modules (named as compared to static frozen modules that cannot be changed without re-flashing MicroPython firmware). Dynamic frozen modules are stored on sectors immediately after the main firmware on the 1st 1MB ROM-mapped flash. Importing DFM takes slightly (about 10%) more RAM than static frozen module, but it takes about 40-45% less RAM than importing .mpy files, which in turn, takes less than half of the memory than importing .py files (because compiling .py files consumes huge amount of memory (5-10 times of the .mpy file size)).
Functions added to the esp module for managing DFMs:
- esp.add_frozen(file_name, module_name) : add a DFM (with <file_name>) as a module (of name <module_name>) to ROM from an
.mpyfile on the file-system; if already exists, overwrite. - esp.del_frozen(module_name): delete a DFM from ROM
- esp.ls_frozen(): list DFMs in the ROM
- esp.reset_frozen(): delete all DFMs in the ROM
Take note: when a module's .mpy file exists both on the filesystem and in the ROM, the version on the filesystem will be loaded, so more memory will be consumed. To avoid that, delete the .mpy file from the filesystem or move it to a different folder or rename it.