This commit improves get handling by guarding against implicit unknown
symbols accessed directly by specific JS native APIs.
Fixes issue #17657.
Signed-off-by: Andrea Giammarchi <andrea.giammarchi@gmail.com>
JavaScript code uses "Symbol in object" to brand check its own proxies, and
such checks should also work on the Python side.
Signed-off-by: Andrea Giammarchi <andrea.giammarchi@gmail.com>
This commit renames the NORETURN macro, indicating to the compiler
that a function does not return, into MP_NORETURN to maintain the same
naming convention of other similar macros.
To maintain compaitiblity with existing code NORETURN is aliased to
MP_NORETURN, but it is also deprecated for MicroPython v2.
This changeset was created using a similar process to
decf8e6a8b ("all: Remove the "STATIC"
macro and just use "static" instead."), with no documentation or python
scripts to change to reflect the new macro name.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
Previously to this commit, running the test suite on a bare-metal board
required specifying the target (really platform) and device, eg:
$ ./run-tests.py --target pyboard --device /dev/ttyACM1
That's quite a lot to type, and you also need to know what the target
platform is, when a lot of the time you either don't care or it doesn't
matter.
This commit makes it easier to run the tests by replacing both of these
options with a single `--test-instance` (`-t` for short) option. That
option specifies the executable/port/device to test. Then the target
platform is automatically detected.
The `--test-instance` can be passed:
- "unix" (the default) to use the unix version of MicroPython
- "webassembly" to test the webassembly port
- anything else is considered a port/device to pass to Pyboard
There are also some shortcuts to specify a port/device, following
`mpremote`:
- a<n> is short for /dev/ttyACM<n>
- u<n> is short for /dev/ttyUSB<n>
- c<n> is short for COM<n>
For example:
$ ./run-tests.py -t a1
Note that the default test instance is "unix" and so this commit does not
change the standard way to run tests on the unix port, by just doing
`./run-tests.py`.
As part of this change, the platform (and it's native architecture if it
supports importing native .mpy files) is show at the start of the test run.
Signed-off-by: Damien George <damien@micropython.org>
This commit makes it so that PyProxy objects are reused (on the JavaScript
side) when they correspond to an existing Python object that is the same
object.
For example, proxying the same Python function to JavaScript, the same
PyProxy instance is now used. This means that if `foo` is a Python
function then accessing it on the JavaScript side such as
`api.globals().get("foo")` has the property that:
api.globals().get("foo") === api.globals().get("foo")
Prior to this commit the above was not true because new PyProxy instances
were created each time `foo` was accessed.
Signed-off-by: Damien George <damien@micropython.org>
In JavaScript when accessing an attribute such as `obj.attr` a value of
`undefined` is returned if the attribute does not exist. This is unlike
Python semantics where an `AttributeError` is raised. Furthermore, in some
cases in JavaScript (eg a Proxy instance) `attr in obj` can return false
yet `obj.attr` is still valid and returns something other than `undefined`.
So the source of truth for whether a JavaScript attribute exists is to just
right away attempt `obj.attr`.
To more closely match these JavaScript semantics when proxying a JavaScript
object through to Python, change the attribute lookup logic on a `JsProxy`
so that it immediately attempts `obj.attr` instead of first testing if the
attribute exists via `attr in obj`.
This allows JavaScript objects which dynamically create attributes to work
correctly on the Python side, with both `obj.attr` and `obj["attr"]`. Note
that `obj["attr"]` already works in all cases because it immediately does
the subscript access without first testing if the attribute exists.
As a benefit, this new behaviour matches the Pyodide behaviour.
Signed-off-by: Damien George <damien@micropython.org>
This allows increasing the Python recursion depth if needed.
Also increase the default to 2k words. There is enough RAM in the
browser/node context for this to be increased, and having a larger pystack
allows more complex code to run without hitting the limit.
Signed-off-by: Damien George <damien@micropython.org>
In the webassembly port there is no asyncio run loop running at the top
level. Instead the Python asyncio run loop is scheduled through setTimeout
and run by the outer JavaScript event loop. Because tasks can become
runable from an external (to Python) event (eg a JavaScript callback), the
run loop must be scheduled whenever a task is pushed to the asyncio task
queue, otherwise tasks may be waiting forever on the queue.
Signed-off-by: Damien George <damien@micropython.org>
This change allows doing a top-level await on an asyncio primitive like
Task and Event.
This feature enables a better interaction and synchronisation between
JavaScript and Python, because `api.runPythonAsync` can now be used (called
from JavaScript) to await on the completion of asyncio primitives.
Signed-off-by: Damien George <damien@micropython.org>
This allows querying the GC heap size/used/free values, as well as the
number of alive JsProxy and PyProxy objects, referenced by proxy_c_ref and
proxy_js_ref.
Signed-off-by: Damien George <damien@micropython.org>
And clear the corresponding `proxy_c_ref[c_ref]` entry when the finaliser
runs. This then allows the C side to (eventually) garbage collect the
corresponding Python object.
Signed-off-by: Damien George <damien@micropython.org>
And clear the corresponding `proxy_js_ref[js_ref]` entry when the finaliser
runs. This then allows the JavaScript side to (eventually) free the
corresponding JavaScript object.
Signed-off-by: Damien George <damien@micropython.org>
So it's possible to know when an external C function is being called at the
top-level, eg by JavaScript without any intermediate C->JS->C calls.
Signed-off-by: Damien George <damien@micropython.org>
Instead of raising KeyError. These semantics match JavaScript behaviour
and make it much more seamless to pass Python dicts through to JavaScript
as though they were JavaScript {} objects.
Signed-off-by: Damien George <damien@micropython.org>
This adds a new undefined singleton to Python, that corresponds directly to
JavaScript `undefined`. It's accessible via `js.undefined`.
Signed-off-by: Damien George <damien@micropython.org>
This reverts part of commit fa23e4b093, to
make it so that Python `None` converts to JavaScript `null` (and JavaScript
`null` already converts to Python `None`). That's consistent with how the
`json` module converts these values back and forth.
Signed-off-by: Damien George <damien@micropython.org>
And change Py None conversion so it converts to JS undefined.
The semantics for conversion of these objects are then:
- Python None -> JavaScript undefined
- JavaScript undefined -> Python None
- JavaScript null -> Python None
This follows Pyodide:
https://pyodide.org/en/stable/usage/type-conversions.html
Signed-off-by: Damien George <damien@micropython.org>
This commit defines a new `JsException` exception type which is used on the
Python side to wrap JavaScript errors. That's then used when a JavaScript
Promise is rejected, and the reason is then converted to a `JsException`
for the Python side to handle.
This new exception is exposed as `jsffi.JsException`.
Signed-off-by: Damien George <damien@micropython.org>
JavaScript semantics are such that the caller of an async function does not
need to await that function for it to run to completion. This commit makes
that behaviour also apply to top-level async Python code run via
`runPythonAsync()`.
Signed-off-by: Damien George <damien@micropython.org>
The `reason` in a rejected promise should be an instance of `Error`. That
leads to better error messages on the JavaScript side.
Signed-off-by: Damien George <damien@micropython.org>
This allows a simple way to run the existing asyncio tests under the
webassembly port, which doesn't support `asyncio.run()`.
Signed-off-by: Damien George <damien@micropython.org>
This commit adds a significant portion of the existing MicroPython asyncio
module to the webassembly port, using parts of the existing asyncio code
and some custom JavaScript parts.
The key difference to the standard asyncio is that this version uses the
JavaScript runtime to do the actual scheduling and waiting on events, eg
Promise fulfillment, timeouts, fetching URLs.
This implementation does not include asyncio.run(). Instead one just uses
asyncio.create_task(..) to start tasks and then returns to the JavaScript.
Then JavaScript will run the tasks.
The implementation here tries to reuse as much existing asyncio code as
possible, and gets all the semantics correct for things like cancellation
and asyncio.wait_for. An alternative approach would reimplement Task,
Event, etc using JavaScript Promise's. That approach is very difficult to
get right when trying to implement cancellation (because it's not possible
to cancel a JavaScript Promise).
Signed-off-by: Damien George <damien@micropython.org>
When a Promise is rejected on the JavaScript side, the reject reason should
be thrown into the encapsulating generator on the Python side.
Signed-off-by: Damien George <damien@micropython.org>
An exception on the Python side should be passed to the Promise reject
callback on the JavaScript side.
Signed-off-by: Damien George <damien@micropython.org>
Otherwise Emscripten allocates it on the Emscripten C stack, which will
overflow for large amounts of code.
Fixes issue #14307.
Signed-off-by: Damien George <damien@micropython.org>
In modularize mode, the `_createMicroPythonModule()` constructor must be
await'ed on, before `Module` is ready to use.
Signed-off-by: Damien George <damien@micropython.org>
This optimises the case where a Python function is, for example, stored to
a JavaScript attribute and then later retrieved from Python. The Python
function no longer needs to be a proxy with double proxying needed for the
call from Python -> JavaScript -> Python.
Signed-off-by: Damien George <damien@micropython.org>
This commit adds a pyscript variant for use in https://pyscript.net/.
The configuration is:
- No ASYNCIFY, in order to keep the WASM size down and have good
performance.
- MICROPY_CONFIG_ROM_LEVEL_FULL_FEATURES to enable most features.
- Custom manifest that includes many of the python-stdlib libraries.
- MICROPY_GC_SPLIT_HEAP_AUTO to increase GC heap size instead of doing a
collection when memory is exhausted. This is needed because ASYNCIFY is
disabled. Instead the GC collection is run at the top-level before
executing any Python code.
- No MICROPY_VARIANT_ENABLE_JS_HOOK because there is no asynchronous
keyboard input to interrupt a running script.
Signed-off-by: Damien George <damien@micropython.org>
With this commit, `interpreter.runPythonAsync(code)` can now be used to run
Python code that uses `await` at the top level. That will yield up to
JavaScript and produce a thenable, which the JavaScript runtime can then
resume. Also implemented is the ability for Python code to await on
JavaScript promises/thenables. For example, outer JavaScript code can
await on `runPythonAsync(code)` which then runs Python code that does
`await js.fetch(url)`. The entire chain of calls will be suspended until
the fetch completes.
Signed-off-by: Damien George <damien@micropython.org>
This commit improves the webassembly port by adding:
- Proxying of Python objects to JavaScript with a PyProxy type that lives
on the JavaScript side. PyProxy implements JavaScript Proxy traps such
as has, get, set and ownKeys, to make Python objects have functionality
on the JavaScript side.
- Proxying of JavaScript objects to Python with a JsProxy type that lives
on the Python side. JsProxy passes through calls, attributes,
subscription and iteration from Python to JavaScript.
- A top-level API on the JavaScript side to construct a MicroPython
interpreter instance via `loadMicroPython()`. That function returns an
object that can be used to execute Python code, access the Python globals
dict, access the Emscripten filesystem, and other things. This API is
based on the API provided by Pyodide (https://pyodide.org/). As part of
this, the top-level file is changed from `micropython.js` to
`micropython.mjs`.
- A Python `js` module which can be used to access all JavaScript-side
symbols, for example the DOM when run within a browser.
- A Python `jsffi` module with various helper functions like
`create_proxy()` and `to_js()`.
- A dedenting lexer which automatically dedents Python source code if every
non-empty line in that source starts with a common whitespace prefix.
This is very helpful when Python source code is indented within a string
within HTML or JavaScript for formatting reasons.
Signed-off-by: Damien George <damien@micropython.org>
