mpremote fails with UnicodeEncodeError on Windows legacy consoles (cp1252)
Port, board and/or hardware
Windows (any hardware) - affects mpremote tool when run on Windows with legacy console (cp1252 or similar encoding).
MicroPython version
- mpremote 1.27.0
- Python 3.11.9 /3.13.1 (host)
- Tested against MicroPython 1.27.0 RP2, ESP32
Reproduction
This issue only occurs with legacy Windows consoles that use cp1252 or similar encodings. Modern terminals (Windows Terminal, VS Code) use UTF-8 by default and are not affected.
To reproduce, you must use a legacy console:
-
Open legacy cmd.exe (not Windows Terminal) or configure PowerShell to use cp1252:
# Force legacy encoding in Python $env:PYTHONIOENCODING = "cp1252" -
Create test files with non-ASCII characters:
mkdir unicode_test echo "test" > "unicode_test\Владимир_Петров.txt" -
Use mpremote to copy:
mpremote cp -rv unicode_test :
Alternatively, the issue can be demonstrated with this Python snippet:
import sys
sys.stdout.reconfigure(encoding='cp1252')
print('Владимир_Петров.txt') # Raises UnicodeEncodeError
Expected behaviour
mpremote cp should successfully copy files with Unicode characters in filenames on Windows. The tool should handle all Unicode characters in console output without raising encoding errors.
CPython's print() should use UTF-8 or properly handle the console encoding.
Observed behaviour
The command fails immediately with:
UnicodeEncodeError: 'charmap' codec can't encode characters in position X-Y: character maps to <undefined>
The file is not copied.
Additional Information
Workaround
Set the PYTHONIOENCODING environment variable before running mpremote:
$env:PYTHONIOENCODING = "utf-8"
mpremote connect COM3 cp -r . :
Or add to PowerShell $PROFILE for persistence:
$env:PYTHONIOENCODING = "utf-8"
$env:PYTHONUTF8 = "1"
Suggested Fix
Force UTF-8 encoding in mpremote on Windows:
import sys
import io
if sys.platform == 'win32':
sys.stdout = io.TextIOWrapper(sys.stdout.buffer, encoding='utf-8', errors='replace')
sys.stderr = io.TextIOWrapper(sys.stderr.buffer, encoding='utf-8', errors='replace')
This would be a minimal change in the mpremote initialization code.
Code of Conduct
Yes, I agree
mpremote cp fails with apostrophe in destination filename
Port, board and/or hardware
Any platform (Windows, Linux, macOS) - affects mpremote tool when copying files with apostrophe (') in the filename.
MicroPython version
- mpremote 1.27.0
- Tested against MicroPython unix port and ESP32
The issue is in mpremote itself, not in MicroPython firmware.
Reproduction
-
Create a test file with an apostrophe in the name:
mkdir unicode_test echo "test" > "unicode_test\O'zbek_Ismoilov.txt" -
Attempt to copy to MicroPython device with the same destination name:
mpremote cp "unicode_test\O'zbek_Ismoilov.txt" ":/O'zbek_Ismoilov.txt" -
Observe the error.
Expected behaviour
mpremote cp should successfully copy files with apostrophes in filenames. The apostrophe should be properly escaped when constructing Python commands to send to the MicroPython REPL.
Observed behaviour
The command fails with a SyntaxError because the apostrophe breaks the Python string literal sent to the REPL.
PS D:\mypython\unicode_mpy> mpremote cp "unicode_test\O'zbek_Ismoilov.txt" ":/O'zbek_Ismoilov.txt"
cp unicode_test\O'zbek_Ismoilov.txt :/O'zbek_Ismoilov.txt
mpremote: Error with transport:
Traceback (most recent call last):
File "<stdin>", line 1
SyntaxError: invalid syntax
Additional Information
mpremote sends filenames to MicroPython's REPL as Python string literals using single quotes. When a filename contains an apostrophe ', it prematurely closes the string:
# What mpremote probably sends:
open('/O'zbek_Ismoilov.txt', 'wb')
# ^ This closes the string prematurely!
# Result is invalid Python syntax
Affected Characters
Any filename containing:
- Single quote / apostrophe:
'(U+0027) - Possibly other quote-like characters that need escaping
Suggested Fix
Properly escape quotes when constructing Python commands to send to the REPL:
Option A: Escape single quotes
# Instead of:
cmd = f"open('{filename}', 'wb')"
# Use:
escaped_filename = filename.replace("\\", "\\\\").replace("'", "\\'")
cmd = f"open('{escaped_filename}', 'wb')"
Option B: Use double quotes for filenames
cmd = f'open("{filename}", "wb")'
# Note: would then need to escape double quotes in filenames
Option C: Use repr() for proper escaping
cmd = f"open({repr(filename)}, 'wb')"
Found via : https://github.com/Josverl/unicode_mpy
Code of Conduct
Yes, I agree