803 Commits

Author SHA1 Message Date
Alessandro Gatti
b63e528076 examples/natmod/btree: Fix build on RV32 with Picolibc.
This commit fixes building the "btree" example natmod on RV32 when
Picolibc is being used and uses thread-local storage for storing the
errno variable.

The fix is surprisingly simple: Picolibc allows overriding the function
that will provide a pointer to the "errno" variable, and the btree
natmod integration code already has all of this machinery set up as part
of its library integration.  Redirecting Picolibc to the already
existing pointer provider function via a compile-time definition is
enough to let the module compile and pass QEMU tests.

This workaround will work on any Picolibc versions (Arm, RV32, Xtensa,
etc.) even if TLS support was not enabled to begin with, and will
effectively do nothing if the toolchain used will rely on Newlib to
provide standard C library functions.

Given that the btree module now builds and passes the relevant natmod
tests, said module is now part of the QEMU port's natmod testing
procedure, and CI now will build the btree module for RV32 as part to
its checks.

Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
2025-07-20 23:03:40 +10:00
Yanfeng Liu
5f55f8d01a tools/pyboard.py: Align execpty prefix.
This aligns the prefix string in L285 to that in L284 though the
two strings have equal length.

Signed-off-by: Yanfeng Liu <yfliu2008@qq.com>
2025-07-20 22:59:06 +10:00
Angus Gratton
a54b5d9aed unix/variants: Add a 'longlong' variant to test 64-bit bigints in CI.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
2025-07-18 00:11:08 +10:00
Jeff Epler
499bedf7aa tools/ci.sh: Always call apt-get update before apt-get install.
There have been recent build failures in build_renesas_ra_board.  It
appears to be the case that a security update for this package was recently
issued by Ubuntu for CVE-2025-4565 and the buggy version is no longer on
package servers.  However, it is still referred to by the cached apt
metadata in the GitHub runners.

Add `apt-get update` to fix this, and audit for other sites in `ci.sh`
where it might also be necessary.

Signed-off-by: Jeff Epler <jepler@gmail.com>
2025-07-12 22:13:29 +10:00
Damien George
05342b013d tools/mpremote: Support OSError's on targets without errno.
Some checks failed
unix port / macos (push) Has been cancelled
unix port / qemu_mips (push) Has been cancelled
unix port / qemu_arm (push) Has been cancelled
unix port / qemu_riscv64 (push) Has been cancelled
unix port / sanitize_address (push) Has been cancelled
unix port / sanitize_undefined (push) Has been cancelled
webassembly port / build (push) Has been cancelled
windows port / build-vs (Debug, x64, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Debug, x64, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Debug, x86, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Debug, x86, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x64, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, x64, dev, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x64, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x64, standard, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, x64, standard, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x64, standard, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x86, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, x86, dev, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x86, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x86, standard, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, x86, standard, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x86, standard, 2022, [17, 18)) (push) Has been cancelled
windows port / build-mingw (i686, mingw32, dev) (push) Has been cancelled
windows port / build-mingw (i686, mingw32, standard) (push) Has been cancelled
windows port / build-mingw (x86_64, mingw64, dev) (push) Has been cancelled
windows port / build-mingw (x86_64, mingw64, standard) (push) Has been cancelled
windows port / cross-build-on-linux (push) Has been cancelled
zephyr port / build (push) Has been cancelled
Python code lint and formatting with ruff / ruff (push) Has been cancelled
Targets without the `errno` module enabled will not render `OSError`s
with the name of the error.  Instead they just print the numeric error
code.

Add support for such targets by explicitly recognising certain error codes.

Signed-off-by: Damien George <damien@micropython.org>
2025-07-08 21:52:56 +10:00
Jeff Epler
a9801f9960 github/workflows: Use Python 3.11 for unix coverage testing.
This removes the need for an explicit `sys_settrace_features.py.exp` file.

This means that people testing locally will also need to install Python
3.11 in some way, such as with pyenv or uv, and use it during
`make VARIANT=coverage test`, or they will get failures.

When using Python from GitHub actions/setup-python, pip3 can't be wrapped
by sudo, because this invokes the operating system python instead.

Signed-off-by: Jeff Epler <jepler@gmail.com>
2025-07-06 00:08:54 +10:00
Jeff Epler
a8c2b917e2 tools/ci.sh: Increase test timeout to 60s in coverage jobs.
The additional overhead of the settrace profiler means that the
`aes_stress.py` test was running too slowly on GitHub CI.  Double the
timeout to 60 seconds.

Signed-off-by: Jeff Epler <jepler@gmail.com>
2025-07-06 00:08:13 +10:00
Jeff Epler
e415d03e7f github/workflows: Remove the unix "settrace" CI job.
This becomes redundant when the main coverage build includes settrace.

Signed-off-by: Jeff Epler <jepler@gmail.com>
2025-07-06 00:07:14 +10:00
Andrew Leech
3a97175f5f tools/mpremote: Fix disconnect handling on Windows and Linux.
Changes in this commit:
- Handle SerialException on Windows when device disconnects.
- Print clean 'device disconnected' message instead of stack trace.
- Fix terminal formatting issues on Linux after disconnect.
- Return disconnected state after console cleanup to avoid terminal issues.

This ensures proper disconnect messages on both platforms without showing
confusing error traces to users.

Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
2025-07-04 15:51:47 +10:00
Jeff Epler
07c3bf21f2 tools/ci.sh: Disable "stack use after return" in ASan build.
This check, runtime-enabled by default in gcc 13 (and existing at least
since gcc 12, but runtime-disabled) changes the stack layout in ways that
are not compatible with assumptions spread across the core code (nlr, gc,
and stack checking).

Signed-off-by: Jeff Epler <jepler@gmail.com>
2025-07-01 15:18:29 +10:00
Damien George
77c454829f tools/autobuild: Build alif boards as part of auto-build.
Signed-off-by: Damien George <damien@micropython.org>
2025-06-26 13:25:37 +10:00
Jeff Epler
268264fe64 tools/ci.sh: Add functions for sanitizer builds.
Includes both undefined and address sanitizer configurations.

Signed-off-by: Jeff Epler <jepler@gmail.com>
2025-06-16 23:28:08 +10:00
Damien George
d9467b07cb tools/mpremote: Improve df command to use new no-arg vfs.mount() query.
The existing `mpremote df` command is not very good, because it needs to
assume that all directories in the root directory are mount points, and
also doesn't correctly stat filesystems when the current directory is not
the root. This leads to wrong results

With the introduction of `vfs.mount()` to return a list of mounted
filesystems and their path, a much better df can be implemented, as done in
this commit.

The new df will also fall back to using the old approach of listing the
root directory if the no-arg `vfs.mount()` query is not supported.

Signed-off-by: Damien George <damien@micropython.org>
2025-06-16 15:11:24 +10:00
Damien George
17951cee87 py/dynruntime.mk: Enable single-precision float by default on armv6/7m.
Soft float now works on these ARM targets thanks to the parent commit.

Signed-off-by: Damien George <damien@micropython.org>
2025-06-10 13:43:03 +10:00
Damien George
718ff4fdd5 tools/mpy_ld.py: Support R_ARM_ABS32 relocation in text.
Add support for R_ARM_ABS32 relocations in native .mpy files.  These can be
rewritten in the same way that data relocations are.

Fixes issue #14430.

Signed-off-by: Damien George <damien@micropython.org>
2025-06-10 13:40:11 +10:00
Alessandro Gatti
0da22b2c30 tools/ci.sh: Fix nanbox CI test runs.
This commit fixes CI test runs for the `nanbox` target, which were
broken by the unconditional native emitter code output changes in the
test runner.

The `nanbox` configuration does not enable native emitters of any kind,
and with a full test run that includes executing emitted native code
things would break when doing CI runs.

This is worked around by introducing a common subset of tests that do
not involve the native emitter, and a more comprehensive set of tests
that include both non-emitter and emitter tests.  The `nanbox` CI test
run will stop at the first subset, whilst other configurations will run
that and execute further tests.

Function names have been kept the same for steps that involve native
code, with the `nanbox` subset having another one.  This should not
trigger any breakage in existing CI configurations or external scripts.

Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
2025-06-10 11:29:02 +10:00
Andrew Leech
95203ab88b tools/boardgen.py: Ensure board pin locals_dict has consistent order.
`tools/boardgen.py` is used by the `make-pins.py` scripts in many ports to
generate the pin definitions for the machine module.

In #17391 it was found that this is currently generating the C structs for
board pin definitions with inconsistent ordering (across different build
runs), which makes it sometimes impossible to get a consistent binary file
even for no change in source files.

This commit fixes that by sorting the board pin names alphabetically.

Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
2025-06-10 11:09:14 +10:00
Alessandro Gatti
b8e56a17b1 github/workflows: Split QEMU/Arm builds into separate entries.
This commit takes the QEMU/Arm CI build and test step and splits it into
three separate steps (bigendian, sabrelite, thumb), to allow them to run
in parallel.

Currently the QEMU/Arm CI build step would take up to 16 minutes, often
being the last step blocking a full test run.  With this commit, when
the steps run in parallel the time it takes to complete the QEMU/Arm
build and test procedure is cut in half - taking between 8 to 9 minutes
depending on the CI runner load.

The existing `ci_build_and_test_arm` function has been removed, in
favour of having three separate functions - one per configuration.  They
are called `ci_build_and_test_arm_bigendian`,
`ci_build_and_test_arm_sabrelite`, and `ci_build_and_test_arm_thumb`.

Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
2025-06-10 10:57:32 +10:00
Damien George
4c55b0879b tools/ci.sh: Allow errors in code-size build to fail the CI.
It was possible for CI to pass even if the bare-arm port fails to build.
This commit fixes that.

Signed-off-by: Damien George <damien@micropython.org>
2025-06-05 14:57:23 +10:00
Alessandro Gatti
193603dbac tools/ci.sh: Clean the correct MPY files when batch compiling.
Some checks failed
unix port / settrace (push) Has been cancelled
unix port / settrace_stackless (push) Has been cancelled
unix port / macos (push) Has been cancelled
unix port / qemu_mips (push) Has been cancelled
unix port / qemu_arm (push) Has been cancelled
unix port / qemu_riscv64 (push) Has been cancelled
webassembly port / build (push) Has been cancelled
windows port / build-vs (Debug, x64, windows-2022, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Debug, x64, windows-latest, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Debug, x86, windows-2022, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Debug, x86, windows-latest, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, x64, windows-2019, dev, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x64, windows-2019, standard, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x64, windows-2022, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x64, windows-2022, standard, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x64, windows-latest, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, x64, windows-latest, standard, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, x86, windows-2019, dev, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x86, windows-2019, standard, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x86, windows-2022, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x86, windows-2022, standard, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x86, windows-latest, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, x86, windows-latest, standard, 2017, [15, 16)) (push) Has been cancelled
windows port / build-mingw (i686, mingw32, dev) (push) Has been cancelled
windows port / build-mingw (i686, mingw32, standard) (push) Has been cancelled
windows port / build-mingw (x86_64, mingw64, dev) (push) Has been cancelled
windows port / build-mingw (x86_64, mingw64, standard) (push) Has been cancelled
windows port / cross-build-on-linux (push) Has been cancelled
zephyr port / build (push) Has been cancelled
Python code lint and formatting with ruff / ruff (push) Has been cancelled
This commit fixes a small yet harmless issue that occurs when invoking
`ci_native_mpy_modules_build` on a persistent environment, as only X64
MPY files would be removed by the cleaning process.

Now the correct architecture is passed at all times when cleaning before
building a natmod for a particular architecture, forcing a full build of
all files to better simulate the CI environment (where there's no state
persisted between runs for this step).

Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
2025-06-04 22:35:40 +10:00
Alessandro Gatti
e8c92240e2 tools/ci.sh: Remove natmod build restrictions for Xtensa.
This commit lets the CI pipeline build all natmods for the Xtensa
target, now that ROM symbols can be used in the linking process.

The restriction was put in place due to build failures on certain
natmods for Xtensa, as ROM symbols would not be used, causing undefined
symbol errors at build time.

Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
2025-06-04 22:35:39 +10:00
Alessandro Gatti
bf2005de9e tools/mpy_ld.py: Resolve fixed-address symbols if requested.
This commit lets mpy_ld.py resolve symbols not only from the object
files involved in the linking process, or from compiler-supplied static
libraries, but also from a list of symbols referenced by an absolute
address (usually provided by the system's ROM).

This is needed for ESP8266 targets as some C stdlib functions are
provided by the MCU's own ROM code to reduce the final code footprint,
and therefore those functions' implementation was removed from the
compiler's support libraries.  This means that unless `LINK_RUNTIME` is
set (which lets tooling look at more libraries to resolve symbols) the
build process will fail as tooling is unaware of the ROM symbols'
existence.  With this change, fixed-address symbols can be exposed to
the symbol resolution step when performing natmod linking.

If there are symbols coming in from a fixed-address symbols list and
internal code or external libraries, the fixed-address symbol address
will take precedence in all cases.

Although this is - in theory - also working for the whole range of ESP32
MCUs, testing is currently limited to Xtensa processors and the example
natmods' makefiles only make use of this commit's changes for the
ESP8266 target.

Natmod builds can set the MPY_EXTERN_SYM_FILE variable pointing to a
linkerscript file containing a series of symbols (weak or strong) at a
fixed address; these symbols will then be used by the MicroPython
linker when packaging the natmod.  If a different natmod build method is
used (eg. custom CMake scripts), `tools/mpy_ld.py` can now accept a
command line parameter called `--externs` (or its short variant `-e`)
that contains the path of a linkerscript file with the fixed-address
symbols to use when performing the linking process.

The linkerscript file parser can handle a very limited subset of
binutils's linkerscript syntax, namely just block comments, strong
symbols, and weak symbols.  Each symbol must be in its own line for the
parser to succeed, empty lines or comment blocks are skipped.  For an
example of what this parser was meant to handle, you can look at
`ports/esp8266/boards/eagle.rom.addr.v6.ld` and follow its format.

The natmod developer documentation is also updated to reflect the new
command line argument accepted by `mpy_ld.py` and the use cases for the
changes introduced by this commit.

Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
2025-06-04 22:35:39 +10:00
Damien George
dea3035b88 tools/pyboard.py: Add write_timeout and catch errors in enter_raw_repl.
Some checks failed
unix port / settrace (push) Has been cancelled
unix port / settrace_stackless (push) Has been cancelled
unix port / macos (push) Has been cancelled
unix port / qemu_mips (push) Has been cancelled
unix port / qemu_arm (push) Has been cancelled
unix port / qemu_riscv64 (push) Has been cancelled
webassembly port / build (push) Has been cancelled
windows port / build-vs (Debug, x64, windows-2022, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Debug, x64, windows-latest, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Debug, x86, windows-2022, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Debug, x86, windows-latest, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, x64, windows-2019, dev, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x64, windows-2019, standard, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x64, windows-2022, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x64, windows-2022, standard, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x64, windows-latest, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, x64, windows-latest, standard, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, x86, windows-2019, dev, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x86, windows-2019, standard, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x86, windows-2022, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x86, windows-2022, standard, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x86, windows-latest, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, x86, windows-latest, standard, 2017, [15, 16)) (push) Has been cancelled
windows port / build-mingw (i686, mingw32, dev) (push) Has been cancelled
windows port / build-mingw (i686, mingw32, standard) (push) Has been cancelled
windows port / build-mingw (x86_64, mingw64, dev) (push) Has been cancelled
windows port / build-mingw (x86_64, mingw64, standard) (push) Has been cancelled
windows port / cross-build-on-linux (push) Has been cancelled
zephyr port / build (push) Has been cancelled
Python code lint and formatting with ruff / ruff (push) Has been cancelled
If the USB serial device locks up, then writes to that device can hang
forever.  That can make the test runner `tests/run-tests.py` lock up, among
other problems.

This commit introduces a 5 second write-timeout, and catches any OSError's
raised during `enter_raw_repl()`.  Now, if a USB serial device locks up,
`enter_raw_repl()` will eventually raise an exception.

Signed-off-by: Damien George <damien@micropython.org>
2025-06-02 17:02:59 +10:00
Damien George
b1c947ab45 tools/pyboard.py: Introduce timeout_overall for read_until().
This applies the mpremote commit 03fe9c55ea
to pyboard.py.

The `timeout_overall` is used in `enter_raw_repl()`.  It prevents waiting
forever for a serial device that does not respond to the Ctrl-C/Ctrl-D/etc
commands and is constantly outputting data.

Signed-off-by: Damien George <damien@micropython.org>
2025-06-02 17:02:30 +10:00
Damien George
aaedd59b7c tools/pyboard.py: Avoid initial blocking read in read_until().
This applies the mpremote commit 0d46e45a1f
to pyboard.py.

If the target does not return any data then `read_until()` will block
indefinitely.  Fix this by making the initial read part of the general read
look, which always checks `inWaiting() > 0` before reading from the serial
device.

Also added the UART timeout to the constructor.  This is not currently used
but may be used as an additional safeguard.

Signed-off-by: Damien George <damien@micropython.org>
2025-06-02 17:02:30 +10:00
Jos Verlinde
d945316432 tools/mpremote/tests: Add tests for 'fs tree' command.
Signed-off-by: Jos Verlinde <jos_verlinde@hotmail.com>
2025-05-18 00:20:43 +10:00
Jos Verlinde
1dfb5092fc tools/mpremote: Add new 'fs tree' command.
Add `mpremote fs tree` command to show a tree of the device's files.  It:
- Shows a treeview from current path or specified path.
- Uses the graph chars ("├── ", "└── ") (not configurable).
- Has the options:
    -v/--verbose adds the serial device name to the top of the tree
    -s/--size add a size to the files
    -h/--human add a human readable size to the files

Signed-off-by: Jos Verlinde <jos_verlinde@hotmail.com>
2025-05-18 00:19:14 +10:00
IhorNehrutsa
f48b981567 esp32: Update to use ESP-IDF v5.4.1.
This version of the IDF uses about 1KB more IRAM and 1KB more DRAM on most
boards, but 6.5KB more DRAM usage on the S3.  It seems that's due to a lot
of small increases in many components.

Signed-off-by: Ihor Nehrutsa <Ihor.Nehrutsa@gmail.com>
2025-05-16 14:02:39 +10:00
Jeff Epler
6d74b4e3c1 tools/gen-cpydiff.py: Ensure every item has at least 2 TOC levels.
Some checks failed
unix port / settrace (push) Has been cancelled
unix port / settrace_stackless (push) Has been cancelled
unix port / macos (push) Has been cancelled
unix port / qemu_mips (push) Has been cancelled
unix port / qemu_arm (push) Has been cancelled
unix port / qemu_riscv64 (push) Has been cancelled
webassembly port / build (push) Has been cancelled
windows port / build-vs (Debug, x64, windows-2022, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Debug, x64, windows-latest, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Debug, x86, windows-2022, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Debug, x86, windows-latest, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, x64, windows-2019, dev, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x64, windows-2019, standard, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x64, windows-2022, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x64, windows-2022, standard, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x64, windows-latest, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, x64, windows-latest, standard, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, x86, windows-2019, dev, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x86, windows-2019, standard, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x86, windows-2022, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x86, windows-2022, standard, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x86, windows-latest, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, x86, windows-latest, standard, 2017, [15, 16)) (push) Has been cancelled
windows port / build-mingw (i686, mingw32, dev) (push) Has been cancelled
windows port / build-mingw (i686, mingw32, standard) (push) Has been cancelled
windows port / build-mingw (x86_64, mingw64, dev) (push) Has been cancelled
windows port / build-mingw (x86_64, mingw64, standard) (push) Has been cancelled
windows port / cross-build-on-linux (push) Has been cancelled
zephyr port / build (push) Has been cancelled
Python code lint and formatting with ruff / ruff (push) Has been cancelled
Previously, the navigation ended up messy when the (long) description of
the item became used as a 2nd level header, meaning that it was placed in
the navigation.  Check for this when generating cpydiff so that new cases
don't sneak in unnoticed.

Signed-off-by: Jeff Epler <jepler@gmail.com>
2025-05-16 11:52:03 +10:00
Jeff Epler
a19d3f742e tools/gen-cpydiff.py: Fix RST heading generation.
The heading character for the difference title was always "~", but items
had been added which had just a single heading level.  This made the
generated table of contents confused about heading levels, because heading
levels are not fixed in rst, but are inferred from the order they appear in
the document.

Signed-off-by: Jeff Epler <jepler@gmail.com>
2025-05-16 11:52:03 +10:00
Jeff Epler
605eda158d tools/gen-cpydiff.py: Improve stdout vs stderr interleaving.
In the syntax_space cpydiff, all the warnings were shown after the other
output.  This is because the output always showed all of stdout first and
all of stdout second.

By running Python in unbuffered mode and using `stderr=STDOUT`, the two
streams are interleaved in exactly the order they're printed, so the
SyntaxWarnings are interleaved with the other output.

By using the `encoding=` argument of Popen, the need to explicitly convert
to utf-8 is avoided.  The encoding of the input also becomes utf-8 in this
case, which all the test cases are (well, they're all ASCII, I think).  As
in `run-tests.py`, setting PYTHONIOENCODING ensures the Python
interpreter's input and output are in utf-8, which is not always the case,
especially on Windows systems.

I spot-checked the generated doc pages and they all seemed to make sense
still.

Signed-off-by: Jeff Epler <jepler@gmail.com>
2025-05-16 11:52:03 +10:00
Daniël van de Giessen
4208970451 tools/verifygitlog.py: Allow long co-author and sign-off names.
Signed-off-by: Daniël van de Giessen <daniel@dvdgiessen.nl>
2025-05-15 13:03:11 +10:00
Damien George
51b821ce82 tools/verifygitlog.py: Disallow a leading slash in commit subject line.
Signed-off-by: Damien George <damien@micropython.org>
2025-05-15 13:01:41 +10:00
Damien George
61eedbbd11 tools/verifygitlog.py: Apply stricter rules on git subject line.
There is a bit of ambiguity as to how the prefix of the git subject line
should look like.  Eg `py/vm: ...` vs `py/vm.c: ...` (whether the extension
should be there or not).

This commit makes the existing CI check of the git commit message stricter,
by applying extra rules to the prefix, the bit before the : in the subject
line.  It now checks that the subject prefix:
- doesn't start with unwanted bits: ., /, ports/
- doesn't have an extension: .c, .h, .cpp, .js, .rst or .md

Full error messages are given when a rule does not pass.

This helps to reduce maintainer burden by applying stricter rules, to keep
the git commit history consistent.

Signed-off-by: Damien George <damien@micropython.org>
2025-05-12 13:47:03 +10:00
Damien George
e53f262a85 tools/mpremote: For mip install, use hash to skip files that exist.
When using `mip install`, if a file that needs to be downloaded already
exists locally, then the hash of that local file will be computed and if it
matches the known hash of the remote file it will not be downloaded.

Hashes in mip are guaranteed unique, so this change should never leave
stale files on the filesystem.

This behaviour follows that of the `mip` package in `micropython-lib`.

Signed-off-by: Damien George <damien@micropython.org>
2025-05-02 11:44:32 +10:00
Damien George
4117a2d9b5 tools/ci.sh: Update URL for xtensa-lx106-elf-standalone.tar.gz.
The https://github.com/jepler/esp-open-sdk repository has been removed, so
use the file hosted at micropython.org (it's the same file).

Signed-off-by: Damien George <damien@micropython.org>
2025-05-02 11:39:49 +10:00
Maureen Helm
3c8d1b13f5 zephyr: Upgrade to Zephyr v4.0.0.
Updates the Zephyr port build instructions. The CI is updated to use
Zephyr docker image 0.27.4, SDK 0.17.0 and the latest Zephyr release
tag.

Tested on max32690fthr and frdm_k64f.

Signed-off-by: Maureen Helm <maureen.helm@analog.com>
Signed-off-by: Detlev Zundel <dzu@member.fsf.org>
2025-04-29 23:00:37 +10:00
Jos Verlinde
6406afb1f3 tools/mpremote: Prevent deletion of /remote files via rm -r.
Some checks failed
unix port / settrace (push) Has been cancelled
unix port / settrace_stackless (push) Has been cancelled
unix port / macos (push) Has been cancelled
unix port / qemu_mips (push) Has been cancelled
unix port / qemu_arm (push) Has been cancelled
unix port / qemu_riscv64 (push) Has been cancelled
webassembly port / build (push) Has been cancelled
windows port / build-vs (Debug, x64, windows-2022, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Debug, x64, windows-latest, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Debug, x86, windows-2022, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Debug, x86, windows-latest, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, x64, windows-2019, dev, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x64, windows-2019, standard, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x64, windows-2022, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x64, windows-2022, standard, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x64, windows-latest, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, x64, windows-latest, standard, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, x86, windows-2019, dev, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x86, windows-2019, standard, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x86, windows-2022, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x86, windows-2022, standard, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x86, windows-latest, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, x86, windows-latest, standard, 2017, [15, 16)) (push) Has been cancelled
windows port / build-mingw (i686, mingw32, dev) (push) Has been cancelled
windows port / build-mingw (i686, mingw32, standard) (push) Has been cancelled
windows port / build-mingw (x86_64, mingw64, dev) (push) Has been cancelled
windows port / build-mingw (x86_64, mingw64, standard) (push) Has been cancelled
windows port / cross-build-on-linux (push) Has been cancelled
zephyr port / build (push) Has been cancelled
Python code lint and formatting with ruff / ruff (push) Has been cancelled
Removes the risk of inadvertently deleting files on the host by preventing
the deletion of files via `rm -r` on the `/remote` vfs mount point.

Fixes issue #17147.

Signed-off-by: Jos Verlinde <jos_verlinde@hotmail.com>
2025-04-26 16:07:44 +10:00
Jos Verlinde
37fe3f66c3 tools/mpremote/tests: Add test for rm -r on /remote vfs.
Signed-off-by: Jos Verlinde <jos_verlinde@hotmail.com>
2025-04-26 16:07:40 +10:00
Anson Mansfield
dc46cf15c1 tools/mpremote: Fix possibly-missing EOPNOTSUPP errno name.
Signed-off-by: Anson Mansfield <amansfield@mantaro.com>
2025-04-26 15:51:49 +10:00
Anson Mansfield
805fe083a3 tools/mpremote: Refactor error handling to apply generally to any errno.
This rewrites the code that previously manually emitted and caught various
OSError subclasses with equivalent code that uses the errno name dictionary
to do this generically, and updates the exception handler in do_filesystem
to catch them in a similarly-generic fashion using os.strerror to retrieve
an appropriate error message text equivalent to the current messages.

Note that in the CPython environments where mpremote runs, the call to the
OSError constructor already returns an instance of the corresponding mapped
exception subtype.

Signed-off-by: Anson Mansfield <amansfield@mantaro.com>
2025-04-26 15:50:00 +10:00
Anson Mansfield
cee0419021 tools/mpremote/tests: Add tests for errno behavior.
Signed-off-by: Anson Mansfield <amansfield@mantaro.com>
2025-04-26 15:49:48 +10:00
Damien George
dcca3ff602 tools/mpremote: Use zlib.compressobj instead of zlib.compress.
Because the `wbits` parameter was only added to `zlib.compress` in
CPython 3.11.  Using `zlib.compressobj` makes the code compatible with much
older CPython versions.

Fixes issue #17140.

Signed-off-by: Damien George <damien@micropython.org>
2025-04-24 22:33:05 +10:00
Angus Gratton
f5cb9eb974 top: Bump Ruff version to v0.11.6.
Brings it into sync with a matching change to micropython-lib (which was
much older).  Includes one small automatic fix.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
2025-04-24 21:42:21 +10:00
Jos Verlinde
ef8282c717 docs/reference/mpremote: Update docs for mpremote rm -r.
Some checks failed
unix port / settrace (push) Has been cancelled
unix port / settrace_stackless (push) Has been cancelled
unix port / macos (push) Has been cancelled
unix port / qemu_mips (push) Has been cancelled
unix port / qemu_arm (push) Has been cancelled
unix port / qemu_riscv64 (push) Has been cancelled
webassembly port / build (push) Has been cancelled
windows port / build-vs (Debug, x64, windows-2022, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Debug, x64, windows-latest, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Debug, x86, windows-2022, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Debug, x86, windows-latest, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, x64, windows-2019, dev, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x64, windows-2019, standard, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x64, windows-2022, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x64, windows-2022, standard, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x64, windows-latest, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, x64, windows-latest, standard, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, x86, windows-2019, dev, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x86, windows-2019, standard, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x86, windows-2022, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x86, windows-2022, standard, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x86, windows-latest, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, x86, windows-latest, standard, 2017, [15, 16)) (push) Has been cancelled
windows port / build-mingw (i686, mingw32, dev) (push) Has been cancelled
windows port / build-mingw (i686, mingw32, standard) (push) Has been cancelled
windows port / build-mingw (x86_64, mingw64, dev) (push) Has been cancelled
windows port / build-mingw (x86_64, mingw64, standard) (push) Has been cancelled
windows port / cross-build-on-linux (push) Has been cancelled
zephyr port / build (push) Has been cancelled
Python code lint and formatting with ruff / ruff (push) Has been cancelled
Signed-off-by: Jos Verlinde <Jos_Verlinde@hotmail.com>
2025-04-09 10:51:48 +10:00
Jos Verlinde
72d4c40941 tools/mpremote/tests: Add tests for mpremote rm -r.
Signed-off-by: Jos Verlinde <Jos_Verlinde@hotmail.com>
2025-04-09 10:51:45 +10:00
Jos Verlinde
1aa9b3d94b tools/mpremote: Add recursive remove functionality to filesystem cmds.
mpremote now supports `mpremote rm -r`.

Addresses #9802 and #16845.

Signed-off-by: Jos Verlinde <Jos_Verlinde@hotmail.com>
2025-04-09 10:44:45 +10:00
iabdalkader
547207ddc8 github/workflows: Add Alif port to CI.
Signed-off-by: iabdalkader <i.abdalkader@gmail.com>
2025-04-09 00:22:33 +10:00
iabdalkader
9e9be83fd6 tools/mpremote: Allow .img for ROMFS file and validate ROMFS image.
Some checks failed
unix port / settrace (push) Has been cancelled
unix port / settrace_stackless (push) Has been cancelled
unix port / macos (push) Has been cancelled
unix port / qemu_mips (push) Has been cancelled
unix port / qemu_arm (push) Has been cancelled
unix port / qemu_riscv64 (push) Has been cancelled
webassembly port / build (push) Has been cancelled
windows port / build-vs (Debug, x64, windows-2022, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Debug, x64, windows-latest, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Debug, x86, windows-2022, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Debug, x86, windows-latest, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, x64, windows-2019, dev, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x64, windows-2019, standard, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x64, windows-2022, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x64, windows-2022, standard, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x64, windows-latest, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, x64, windows-latest, standard, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, x86, windows-2019, dev, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x86, windows-2019, standard, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x86, windows-2022, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x86, windows-2022, standard, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x86, windows-latest, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, x86, windows-latest, standard, 2017, [15, 16)) (push) Has been cancelled
windows port / build-mingw (i686, mingw32, dev) (push) Has been cancelled
windows port / build-mingw (i686, mingw32, standard) (push) Has been cancelled
windows port / build-mingw (x86_64, mingw64, dev) (push) Has been cancelled
windows port / build-mingw (x86_64, mingw64, standard) (push) Has been cancelled
windows port / cross-build-on-linux (push) Has been cancelled
zephyr port / build (push) Has been cancelled
Python code lint and formatting with ruff / ruff (push) Has been cancelled
Currently the tool allows writing an invalid ROMFS image, with a bad header
or images smaller than minimum size, and only checks the image extension.

This commit allows deploying a ROMFS with either a ".img" or ".romfs"
extension (in the future support may be added for other extensions that
have different semantics, eg a manifest), and validates the image header
before writing.

Signed-off-by: iabdalkader <i.abdalkader@gmail.com>
2025-04-08 00:27:55 +10:00
Angus Gratton
74a5bf94c1 tools/gen-cpydiff.py: Fail CPython diff generation if output matches.
Some checks failed
unix port / settrace (push) Has been cancelled
unix port / settrace_stackless (push) Has been cancelled
unix port / macos (push) Has been cancelled
unix port / qemu_mips (push) Has been cancelled
unix port / qemu_arm (push) Has been cancelled
unix port / qemu_riscv64 (push) Has been cancelled
webassembly port / build (push) Has been cancelled
windows port / build-vs (Debug, x64, windows-2022, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Debug, x64, windows-latest, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Debug, x86, windows-2022, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Debug, x86, windows-latest, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, x64, windows-2019, dev, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x64, windows-2019, standard, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x64, windows-2022, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x64, windows-2022, standard, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x64, windows-latest, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, x64, windows-latest, standard, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, x86, windows-2019, dev, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x86, windows-2019, standard, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x86, windows-2022, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x86, windows-2022, standard, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x86, windows-latest, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, x86, windows-latest, standard, 2017, [15, 16)) (push) Has been cancelled
windows port / build-mingw (i686, mingw32, dev) (push) Has been cancelled
windows port / build-mingw (i686, mingw32, standard) (push) Has been cancelled
windows port / build-mingw (x86_64, mingw64, dev) (push) Has been cancelled
windows port / build-mingw (x86_64, mingw64, standard) (push) Has been cancelled
windows port / cross-build-on-linux (push) Has been cancelled
zephyr port / build (push) Has been cancelled
Python code lint and formatting with ruff / ruff (push) Has been cancelled
Previously this information was recorded in a "status" field of the result,
but nothing ever parsed this result which led to non-differences not being
removed.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
2025-04-07 16:10:05 +10:00