[webassembly][pyscript] Unable to process over 88Kb of code
Checks
-
I agree to follow the MicroPython Code of Conduct to ensure a safe and respectful space for everyone.
-
I've searched for existing issues matching this bug, and didn't find any.
Port, board and/or hardware
webassemply
MicroPython version
latest for pyscript
Reproduction
We bootstrap our virtual env via FS operations and we port that whole operation via strings (as in Python strings) but we reached a memory limit so that everything breaks. Under 60Kb of single run we don't have issues but beyond that it's breaking.
Expected behaviour
MicroPython should be able to run up to 256Kb of code or more to me or we should be able to raise that limit. We tried to explicitly increase the heapsize without any success.
Observed behaviour
runPython(moreThan88KCode) fails, it doesn't on Pyodide, it doesn't if we manually remove some line to evaluate from that code.
Additional Information
This file contains our bootstrap logic which is automatically generated and it travels across workers or projects to be sure our namespace is up and running by the time users' code executes.
It served us well as logic to date and we never had issues with MicroPython but a recent branch that would like to bring more is failing hard on it.
[webassembly] [pyscript] Unable to create a File from the virtual FileSystem
Checks
-
I agree to follow the MicroPython Code of Conduct to ensure a safe and respectful space for everyone.
-
I've searched for existing issues matching this bug, and didn't find any.
Port, board and/or hardware
webassembly
MicroPython version
3.4.0; MicroPython v1.23.0-preview.334.g9c7f0659e on 2024-04-25
Reproduction
<!doctype html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<script type="module">
const base = 'https://cdn.jsdelivr.net/npm/@micropython/micropython-webassembly-pyscript@latest';
const { loadMicroPython } = await import(`${base}/micropython.mjs`);
const interpreter = await loadMicroPython({ url: `${base}/micropython.wasm` });
// sync or async has the same No.2 issue
// async also has the No.1 issue though
interpreter.runPythonAsync(`
import js, jsffi
async def after_fetch(request):
name = "page.html"
txt = await request.text()
# for report purpose, stored as text
with open(f"./{name}", "w") as dest:
dest.write(txt)
dest.close()
# but read as generic binary data
with open(f"./{name}", "rb") as source:
data = source.read()
source.close()
buffer = js.Uint8Array.new(len(data))
for pos, b in enumerate(data):
buffer[pos] = b
details = jsffi.to_js({"type": "text/html"})
# Issue No.2 - stringToUTF8Array expects a string (got symbol)
file = js.File.new([buffer], name, details)
tmp = js.URL.createObjectURL(file)
dest = js.document.createElement("a")
dest.setAttribute("download", name)
dest.setAttribute("href", tmp)
dest.click()
js.setTimeout(lambda: js.URL.revokeObjectURL(tmp), 10000)
# Issue No.1 - this fails: silent "nothingness"
# await after_fetch(await js.fetch(js.location.href))
# ... or even just ...
# after_fetch(await js.fetch(js.location.href))
# this works though ... next ...
js.fetch(js.location.href).then(after_fetch)
`);
</script>
</head>
</html>
Expected behaviour
With just File references coming directly from JS world, we have zero issues in creating downloads in the wild:
<!doctype html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<script type="module">
const base = 'https://cdn.jsdelivr.net/npm/@micropython/micropython-webassembly-pyscript@latest';
const { loadMicroPython } = await import(`${base}/micropython.mjs`);
const interpreter = await loadMicroPython({ url: `${base}/micropython.wasm` });
interpreter.runPython(`
import js
def on_change(event):
file, = event.currentTarget.files
tmp = js.URL.createObjectURL(file)
dest = js.document.createElement("a")
dest.setAttribute("download", file.name)
dest.setAttribute("href", tmp)
dest.click()
js.setTimeout(lambda: js.URL.revokeObjectURL(tmp), 10000)
js.document.querySelector('input[type="file"]').onchange = on_change
`);
</script>
</head>
<body>
<input type="file">
</body>
</html>
Select a generic file and see it downloaded too.
However, when files are pre-stored or added to the virtual File System, no matter which dance we do to convert these, the moment we try to create a new File reference, we always get the same error:
Aborted(Assertion failed: stringToUTF8Array expects a string (got symbol))
micropython.mjs:1 Uncaught (in promise) RuntimeError: Aborted(Assertion failed: stringToUTF8Array expects a string (got symbol))
at abort (micropython.mjs:1:11816)
at assert (micropython.mjs:1:8180)
at stringToUTF8Array (micropython.mjs:1:19921)
at stringToUTF8 (micropython.mjs:1:65962)
at micropython.mjs:1:66768
at string (micropython.mjs:1:66800)
at Object.ccall (micropython.mjs:1:67374)
at Object.get (micropython.mjs:1:100012)
at js_reflect_construct (micropython.mjs:1:79716)
at micropython.wasm:0x35fb4
Observed behaviour
We are incapable of downloading files stored in the virtual file system but actually we are incapable of creating any file at all due very same error.
Additional Information
Somehow there's another hidden bug in this report: we are also incapable of awaiting something top level with no warnings or issues whatsoever or no errors at all anywhere if we don't force the js.fetch(url).then(...) instead of just await js.fetc(url) first.