Commit Graph

15979 Commits

Author SHA1 Message Date
Angus Gratton
908ab1ceca py/objint: Fix int.to_bytes() buffer size checks.
Fixes and improvements to `int.to_bytes()` are:
- No longer overflows if byte size is 0 (closes #13041).
- Raises OverflowError in any case where number won't fit into byte length
  (now matches CPython, previously MicroPython would return a truncated
  bytes object).
- Document that `micropython int.to_bytes()` doesn't implement the optional
  signed kwarg, but will behave as if `signed=True` when the integer is
  negative (this is the current behaviour).  Add tests for this also.

Requires changes for small ints, MPZ large ints, and "long long" large
ints.

Adds a new set of unit tests for ints between 32 and 64 bits to increase
coverage of "long long" large ints, which are otherwise untested.

Tested on unix port (64 bit small ints, MPZ long ints) and Zephyr STM32WB
board (32 bit small ints, long long large ints).

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
2024-06-24 14:07:00 +10:00
Angus Gratton
d933210d96 py/misc: Move mp_clz and mp_ctz intrinsics into misc.h.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
2024-06-24 14:06:54 +10:00
Damien George
cebc9b0ae2 tools/mpremote: Fix absolute path usage in remote mounted VFS.
Prior to this fix the current working path in the remote VFS would always
be prepended to the requested path to get the full path, even if the
requested path was already absolute, ie starting with "/".

So `os.chdir("/remote/dir1")` would set the working path to "/dir1/", and
a subsequent call with an absolute path like `os.listdir("/remote/dir2")`
would try to list the directory "/dir1/dir2/".

Fixes issue #15308.

Signed-off-by: Damien George <damien@micropython.org>
2024-06-21 16:46:49 +10:00
Damien George
0619f261a8 tests/basics: Add tests to test repeated throw into the same generator.
Signed-off-by: Damien George <damien@micropython.org>
2024-06-21 16:23:08 +10:00
Damien George
038125be79 py/emitnative: Fix native async with.
The code generating the entry to the finally handler of an async-with
statement was simply wrong for the case of the native emitter.  Among other
things the layout of the stack was incorrect.

This is fixed by this commit.  The setup of the async-with finally handler
is now put in a dedicated emit function, for both the bytecode and native
emitters to implement in their own way (the bytecode emitter is unchanged,
just factored to a function).

With this fix all of the async-with tests now work when using the native
emitter.

Signed-off-by: Damien George <damien@micropython.org>
2024-06-21 16:21:33 +10:00
Damien George
a19214d897 py/emitnative: Place thrown value in dedicated local variable.
A value thrown/injected into a native generator needs to be stored in a
dedicated variable outside `nlr_buf_t`, following the `inject_exc` variable
in `py/vm.c`.

Signed-off-by: Damien George <damien@micropython.org>
2024-06-21 16:21:29 +10:00
Damien George
9dbc787ce8 py/emitndebug: Add native debug emitter.
This emitter prints out pseudo-machine instructions, instead of the usual
output of the native emitter.  It can be enabled on any port via
`MICROPY_EMIT_NATIVE_DEBUG` (make sure other native emitters are disabled)
but the easiest way to use it is with mpy-cross:

    $ mpy-cross -march=debug file.py

Signed-off-by: Damien George <damien@micropython.org>
2024-06-21 16:21:27 +10:00
Damien George
e2ae03e979 py/emitnative: Add more DEBUG_printf statements.
Signed-off-by: Damien George <damien@micropython.org>
2024-06-21 16:06:02 +10:00
Alessandro Gatti
c1882e4866 qemu-riscv: Enable native code generation by default.
This turns on the native RV32IMC code generator for the QEMU-based
RISC-V port, and removes tests that relies on native code generation
from the exclusion list (ie enables these tests).

Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
2024-06-21 15:07:03 +10:00
Alessandro Gatti
3dd1130f6d py/emitnative: Emit better load/store sequences for RISC-V RV32IMC.
Selected load/store code sequences have been optimised for RV32IMC when the
chance to use fewer and smaller opcodes was possible.

Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
2024-06-21 15:06:52 +10:00
Alessandro Gatti
99f5659cf5 mpy-cross: Add RISC-V RV32IMC support in MPY files.
MPY files can now hold generated RV32IMC native code.  This can be
accomplished by passing the `-march=rv32imc` flag to mpy-cross.

Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
2024-06-21 15:06:29 +10:00
Alessandro Gatti
8338f66352 py/asmrv32: Add RISC-V RV32IMC native code emitter.
This adds a native code generation backend for RISC-V RV32I CPUs, currently
limited to the I, M, and C instruction sets.

Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
2024-06-21 15:06:07 +10:00
Angus Gratton
5a778ebc37 tests/thread: Re-enable GC before stress_schedule test ends.
Otherwise GC stays disabled (not re-enabled by soft reset) and later test
runs fail with MemoryError.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
2024-06-21 14:45:59 +10:00
Damien George
407464348d tests/cpydiff: Remove deque difference test.
Because `collections.deque` is now a built-in type in MicroPython.

Signed-off-by: Damien George <damien@micropython.org>
2024-06-21 14:35:43 +10:00
Dan Halbert
5040b13dd4 py/objdeque: Fix deque type flags based on option settings.
This fixes a minor issue in the changes made by
7dff38fdc1: the type flags for deque were
meant to be conditionalized based on MICROPY_PY_COLLECTIONS_DEQUE_ITER, but
the computed conditionalized value wasn't used.

Signed-off-by: Dan Halbert <halbert@halwitz.org>
2024-06-21 14:27:33 +10:00
Damien George
88513d1226 webassembly/api: Allow specifying the pystack size.
This allows increasing the Python recursion depth if needed.

Also increase the default to 2k words.  There is enough RAM in the
browser/node context for this to be increased, and having a larger pystack
allows more complex code to run without hitting the limit.

Signed-off-by: Damien George <damien@micropython.org>
2024-06-20 00:26:08 +10:00
Damien George
13195a678d webassembly/asyncio: Schedule run loop when tasks are pushed to queue.
In the webassembly port there is no asyncio run loop running at the top
level.  Instead the Python asyncio run loop is scheduled through setTimeout
and run by the outer JavaScript event loop.  Because tasks can become
runable from an external (to Python) event (eg a JavaScript callback), the
run loop must be scheduled whenever a task is pushed to the asyncio task
queue, otherwise tasks may be waiting forever on the queue.

Signed-off-by: Damien George <damien@micropython.org>
2024-06-20 00:11:54 +10:00
Damien George
8ac9c8f392 extmod/modasyncio: Add support for a callback on TaskQueue push.
Allows passing in a callback to `TaskQueue()` that is called when something
is pushed on to the queue.

Signed-off-by: Damien George <damien@micropython.org>
2024-06-20 00:11:05 +10:00
Damien George
e9c898cb33 webassembly/asyncio: Support top-level await of asyncio Task and Event.
This change allows doing a top-level await on an asyncio primitive like
Task and Event.

This feature enables a better interaction and synchronisation between
JavaScript and Python, because `api.runPythonAsync` can now be used (called
from JavaScript) to await on the completion of asyncio primitives.

Signed-off-by: Damien George <damien@micropython.org>
2024-06-18 22:23:16 +10:00
Damien George
a053e63914 webassembly/objjsproxy: Implement proxying of JS iterable protocol.
This allows Python to iterate over JavaScript objects that provide
Symbol.iterator.

Signed-off-by: Damien George <damien@micropython.org>
2024-06-18 22:14:34 +10:00
Maureen Helm
2b0e64beaf zephyr: Link MicroPython with the Zephyr kernel library.
Unlike most other Zephyr libraries, libkernel.a is not built as a
whole-archive.

This change also fixes a linker error observed on nucleo_wb55rg while
preparing an upgrade to Zephyr v3.5.0, caused by an undefined reference to
`z_impl_k_busy_wait`.

Signed-off-by: Maureen Helm <maureen.helm@analog.com>
2024-06-18 16:19:07 +10:00
Jason Kridner
c211c05291 zephyr/modsocket: Add socket.recvfrom method.
Signed-off-by: Jason Kridner <jkridner@beagleboard.org>
2024-06-18 16:00:21 +10:00
Jason Kridner
0466560b12 zephyr/modzsensor: Add additional sensor type constants.
Signed-off-by: Jason Kridner <jkridner@beagleboard.org>
2024-06-18 15:54:25 +10:00
Alessandro Gatti
411d66586c extmod/modplatform: Add picolibc to the recognised libcs list.
This adds picolibc to the list of the recognised libc options.

Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
2024-06-17 13:12:54 +10:00
Alessandro Gatti
1b10cb843c github/workflows: Add qemu-riscv port to CI.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
2024-06-17 12:06:09 +10:00
Alessandro Gatti
2d69aab7b3 qemu-riscv: Add new QEMU RV32 port.
This adds a QEMU-based bare metal RISC-V 32 bits port.  For the time being
only QEMU's "virt" 32 bits board is supported, using the ilp32 ABI and the
RV32IMC architecture.

The top-level README and the run-tests.py files are updated for this new
port.

Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
2024-06-17 12:06:09 +10:00
Damien George
d7d77d91be qemu-arm: Clean up header file includes.
Signed-off-by: Damien George <damien@micropython.org>
2024-06-11 21:43:45 +10:00
Damien George
9bfb4cec6e qemu-arm: Add license and copyright to files missing them.
All of these files are first-party code written from scratch as part of
this repository, and were added when the top-level MIT license was active,
so they have an MIT license by default.  Tracing back the git history show
the original authors/source/copyright as follows:

- main.c, mpconfigport.h: copied from the bare-arm port [1].

- test_main.c: added in [2].

- mphalport.h: added in [3] then updated in [4].

- mps2.ld, nrf51.ld, stm32.ld, uart.h: added in [4].

- imx6.ld, uart.c, startup.c: added in [4] and updated in [5].

[1] Commit c557215822 in 2014, the initial
    bare-arm port; see related ee857853d6.
[2] Commit c1c32d65af in 2014, initial
    qemu-arm CI tests.
[3] Commit b0a15aa735 in 2016, enabling
    extmods and their tests.
[4] Commit e7332b0584 in 2018, big refactor.
[5] Commit b84406f313 in 2021, adding
    Cortex-A9 support.

Signed-off-by: Damien George <damien@micropython.org>
2024-06-11 16:32:18 +10:00
Damien George
5903ee561c extmod/modlwip: Consolidate socket.accept timeout logic.
This makes the code a bit simpler to understand for the three cases of
timeout behaviour (-1, 0, non-zero), and eliminates a dependency on the
(slow) `mp_hal_delay_ms(100)` call.

Signed-off-by: Damien George <damien@micropython.org>
2024-06-08 09:02:01 +10:00
Damien George
df0d7e9429 extmod/modlwip: Make socket.connect raise ETIMEDOUT on non-zero timeout.
If the socket timeout is 0 then a failed socket.connect() raises
EINPROGRESS (which is what the lwIP bindings already did), but if the
socket timeout is non-zero then a failed socket.connect() should raise
ETIMEDOUT.  The latter is fixed in this commit.

A test is added for these timeout cases.

Signed-off-by: Damien George <damien@micropython.org>
2024-06-08 09:02:01 +10:00
Damien George
80a4f632ee rp2/cyw43_configport: Make cyw43_delay_ms() a busy loop.
Currently, `cyw43_delay_ms()` calls `mp_hal_delay_ms()` which uses PendSV
to set up a timer and wait for an interrupt, using wfe.  But in the cyw43
initialisation stage PendSV is disabled and so this delay suspends on the
wfe instruction for an indefinite amount of time.

Work around this by changing the implementation of `cyw43_delay_ms()` to a
busy loop.

Fixes issue #15220.

Signed-off-by: Damien George <damien@micropython.org>
2024-06-08 08:43:49 +10:00
Damien George
3c8089d1b1 py/lexer: Support raw f-strings.
Support for raw str/bytes already exists, and extending that to raw
f-strings is easy.  It also reduces code size because it eliminates an
error message.

Signed-off-by: Damien George <damien@micropython.org>
2024-06-06 17:34:28 +10:00
Damien George
a066f2308f py/lexer: Support concatenation of adjacent f-strings.
This is quite a simple and small change to support concatenation of
adjacent f-strings, and improve compatibility with CPython.

Signed-off-by: Damien George <damien@micropython.org>
2024-06-06 14:58:46 +10:00
Alessandro Gatti
d7aa2fe9d7 lib/libm: Define _IEEE_LIBM only if not set.
fdilibm was originally meant to see _IEEE_LIBM defined from outside the
libm code, not it being hardcoded in.  Picolibc assumes this assumption
holds true and attempts to define itself, conflicting with the existing
definition.

Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
2024-06-06 13:01:42 +10:00
Alessandro Gatti
ace08c3978 shared/runtime/semihosting: Add RISC-V semihosting support.
This adds a RISC-V RV32 semihosting implementation, with all defined
system calls exposed to the user.

Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
2024-06-06 12:55:21 +10:00
Alessandro Gatti
90d50ce918 shared/runtime/semihosting: Rename ARM semihosting files.
Make room for RISC-V semihosting code, by renaming the existing
`semihosting.[ch]` files into `semihosting_arm.[ch]`.

Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
2024-06-06 12:55:15 +10:00
Alessandro Gatti
de0e13a9a8 shared/runtime/gchelper: Add RISC-V RV32I native gchelper.
Add native gchelper support for 32 bits RISC-V RV32I targets.

Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
2024-06-06 12:01:00 +10:00
Alessandro Gatti
e6ae699998 py/nlrrv32: Add RISC-V RV32I NLR implementation.
Add custom NLR support for 32 bits RISC-V RV32I targets.

Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
2024-06-06 11:56:58 +10:00
Alessandro Gatti
596f92bf77 tools/makemanifest.py: Generate working code for empty manifests.
When no usable manifest directives are found (as opposed to no manifest
being set in the makefile), non-compiling code was generated for the
empty frozen constants pool block.

Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
2024-06-05 14:07:26 +02:00
Andrew Leech
8809ae7161 shared/tinyusb: Buffer startup CDC data to send to host on connection.
At startup, buffer initial stdout / MicroyPthon banner so that it can be
sent to the host on initial connection of the USB serial port.  This
buffering also works for when the CDC becomes disconnected and the device
is still printing to stdout, and when CDC is reconnected the most recent
part of stdout (depending on how big the internal USB FIFO is) is flushed
to the host.

This change is most obvious when you've first plugged in a MicroPython
device (or hit reset), when it's a board that uses USB (CDC) serial in the
chip itself for the REPL interface.  This doesn't apply to UART going via a
separate USB-serial chip.

The stm32 port already has this buffering behaviour (it doesn't use
TinyUSB) and this commit extends such behaviour to rp2, mimxrt, samd and
renesas-ra ports, which do use TinyUSB.

Signed-off-by: Andrew Leech <andrew@alelec.net>
2024-06-04 17:49:19 +10:00
Andrew Leech
d144f0699d lib/tinyusb: Update to the most recent master.
Signed-off-by: Andrew Leech <andrew@alelec.net>
2024-06-04 17:47:51 +10:00
robert-hh
e138bafbc7 cc3200/mods: Implement network.ipconfig and network.WLAN.ipconfig.
Signed-off-by: robert-hh <robert@hammelrath.com>
2024-06-04 12:45:01 +10:00
Felix Dörre
9ece9f9b52 esp8266/network_wlan: Implement network.ipconfig and WLAN.ipconfig.
Co-authored-by: robert-hh <robert@hammelrath.com>

Signed-off-by: Felix Dörre <felix@dogcraft.de>
2024-06-04 12:45:01 +10:00
Felix Dörre
b555d6ccaf esp32/network_ppp: Implement network.PPP.ipconfig method.
Signed-off-by: Felix Dörre <felix@dogcraft.de>
2024-06-04 12:45:01 +10:00
robert-hh
a71471be66 extmod/network_lwip: Allow using the CIDR notation for addr4.
There was a little omisssion in the code.

Signed-off-by: robert-hh <robert@hammelrath.com>
2024-06-04 12:45:01 +10:00
robert-hh
7e7cc2b427 extmod/network_ninaw10: Implement the ipconfig methods for ninaw10.
This implements network.ipconfig() and network.WLAN.ipconfig() when the
ninaw10 driver is used for WLAN.

Due to a omission in the ninaw10 driver stack, setting the DNS address has
no effect.  But the interface is kept here just in case it's fixed
eventually.

dhcp4 and has_dhcp4 are dummy arguments.  Ninaw10 seems to always use DHCP.

Signed-off-by: robert-hh <robert@hammelrath.com>
2024-06-04 12:45:01 +10:00
Felix Dörre
1f23ab1e3d esp32,mimxrt,stm32: Implement ipconfig() for more network interfaces.
Implements:
- esp32: network.ipconfig()
- esp32: network.LAN.ipconfig()
- esp32: network.WLAN.ipconfig()
- mimxrt: network.LAN.ipconfig()
- stm32: network.LAN.ipconfig()

Signed-off-by: Felix Dörre <felix@dogcraft.de>
2024-06-04 12:45:01 +10:00
Damien George
0e19286c94 tools/metrics.py: Change rp2 board selection to RPI_PICO_W.
This tool is used to compute size differences in the firmware (eg as part
of CI), but it doesn't currently check any firmware that has bare-metal
lwIP/networking, making it hard to see how firmware size changes when
networking related changes are made.

So, change the board selection for the rp2 port to RPI_PICO_W.  Changes in
size to standard RPI_PICO firmware will be very similar to other bare-metal
boards like PYBV10.

Signed-off-by: Damien George <damien@micropython.org>
2024-06-04 10:54:35 +10:00
Phil Howard
932f76c6ba rp2/CMakeLists: Use MICROPY_BOARD_DIR to find pins.csv.
Assuming that ${MICROPY_PORT_DIR}/boards/${MICROPY_BOARD} is equal to
${MICROPY_BOARD_DIR} is not valid, because the latter could point to a path
outside the main MicroPython repository.

Replace this path with the canonical ${MICROPY_BOARD_DIR} so that pins.csv
is correctly located when building against out-of-tree board definitions.

Additionally remove MICROPY_BOARDS_DIR to discourage similar mistakes.

Signed-off-by: Phil Howard <phil@gadgetoid.com>
2024-06-03 16:26:08 +10:00
Angus Gratton
a84c7a0ed9 rp2/modmachine: Selectively leave the USB clocks enabled in lightsleep.
Without this change going to lightsleep stops the USB peripheral clock, and
can lead to either the device going into a weird state or the host deciding
to issue a bus reset.

This change only keeps the USB peripheral clocks enabled if the USB device
is currently active and a host has configured the device.  This means the
USB device continues to respond to host transfers and (presumably) will
even complete pending endpoint transfers.  All other requests are NAKed
while still asleep, but the interaction with the host seems to resume
correctly on wake

Otherwise, if USB is not active or configured by a host, USB clocks are
disabled, the same as before.

With the change, one can issue a `machine.lightsleep(...)` with USB CDC
connected and the USB CDC remains connected during the sleep and resumes
when the lightsleep finishes.

Tested on a RPi Pico, the power consumption is:
- During normal idle at the REPL, about 15.3mA.
- During lightsleep, prior to this change, about 1.35mA.
- During lightsleep, with this change and USB CDC connected, about 3.7mA.

If power consumption should be as low as possible when USB is connected,
one can use `machine.USBDevice` to disable the USB before entering
lightsleep.

As discussed at https://github.com/orgs/micropython/discussions/14401

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
2024-06-03 16:00:52 +10:00