There have been recent build failures in build_renesas_ra_board. It
appears to be the case that a security update for this package was recently
issued by Ubuntu for CVE-2025-4565 and the buggy version is no longer on
package servers. However, it is still referred to by the cached apt
metadata in the GitHub runners.
Add `apt-get update` to fix this, and audit for other sites in `ci.sh`
where it might also be necessary.
Signed-off-by: Jeff Epler <jepler@gmail.com>
This removes the need for an explicit `sys_settrace_features.py.exp` file.
This means that people testing locally will also need to install Python
3.11 in some way, such as with pyenv or uv, and use it during
`make VARIANT=coverage test`, or they will get failures.
When using Python from GitHub actions/setup-python, pip3 can't be wrapped
by sudo, because this invokes the operating system python instead.
Signed-off-by: Jeff Epler <jepler@gmail.com>
The additional overhead of the settrace profiler means that the
`aes_stress.py` test was running too slowly on GitHub CI. Double the
timeout to 60 seconds.
Signed-off-by: Jeff Epler <jepler@gmail.com>
This check, runtime-enabled by default in gcc 13 (and existing at least
since gcc 12, but runtime-disabled) changes the stack layout in ways that
are not compatible with assumptions spread across the core code (nlr, gc,
and stack checking).
Signed-off-by: Jeff Epler <jepler@gmail.com>
This commit fixes CI test runs for the `nanbox` target, which were
broken by the unconditional native emitter code output changes in the
test runner.
The `nanbox` configuration does not enable native emitters of any kind,
and with a full test run that includes executing emitted native code
things would break when doing CI runs.
This is worked around by introducing a common subset of tests that do
not involve the native emitter, and a more comprehensive set of tests
that include both non-emitter and emitter tests. The `nanbox` CI test
run will stop at the first subset, whilst other configurations will run
that and execute further tests.
Function names have been kept the same for steps that involve native
code, with the `nanbox` subset having another one. This should not
trigger any breakage in existing CI configurations or external scripts.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit takes the QEMU/Arm CI build and test step and splits it into
three separate steps (bigendian, sabrelite, thumb), to allow them to run
in parallel.
Currently the QEMU/Arm CI build step would take up to 16 minutes, often
being the last step blocking a full test run. With this commit, when
the steps run in parallel the time it takes to complete the QEMU/Arm
build and test procedure is cut in half - taking between 8 to 9 minutes
depending on the CI runner load.
The existing `ci_build_and_test_arm` function has been removed, in
favour of having three separate functions - one per configuration. They
are called `ci_build_and_test_arm_bigendian`,
`ci_build_and_test_arm_sabrelite`, and `ci_build_and_test_arm_thumb`.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit fixes a small yet harmless issue that occurs when invoking
`ci_native_mpy_modules_build` on a persistent environment, as only X64
MPY files would be removed by the cleaning process.
Now the correct architecture is passed at all times when cleaning before
building a natmod for a particular architecture, forcing a full build of
all files to better simulate the CI environment (where there's no state
persisted between runs for this step).
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit lets the CI pipeline build all natmods for the Xtensa
target, now that ROM symbols can be used in the linking process.
The restriction was put in place due to build failures on certain
natmods for Xtensa, as ROM symbols would not be used, causing undefined
symbol errors at build time.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This version of the IDF uses about 1KB more IRAM and 1KB more DRAM on most
boards, but 6.5KB more DRAM usage on the S3. It seems that's due to a lot
of small increases in many components.
Signed-off-by: Ihor Nehrutsa <Ihor.Nehrutsa@gmail.com>
Updates the Zephyr port build instructions. The CI is updated to use
Zephyr docker image 0.27.4, SDK 0.17.0 and the latest Zephyr release
tag.
Tested on max32690fthr and frdm_k64f.
Signed-off-by: Maureen Helm <maureen.helm@analog.com>
Signed-off-by: Detlev Zundel <dzu@member.fsf.org>
If picotool is not installed, it's fetched and built when compiling each
rp2 board. And the "develop" branch of picotool is used instead of a
release. Installing it manually using the "master" branch means the latest
released version is used (instead of a possibly unstable development
version), and also makes building each rp2 board a little faster.
Signed-off-by: Damien George <damien@micropython.org>
This commit removes the assumption made by the CI scripts that the
system-provided python executable is simply named "python". The scripts
will now look for a binary called "python3" first, and then fall back to
"python" if that is not found.
Whilst this is currently the case for the CI environment, there are no
guarantees for this going forward. For example minimal CI environments
set up by some developers, using the same base OS, have their python
executable called "python3".
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit expands the CI tests by checking whether the example native
modules are able to be built for the Xtensa architecture.
This was made possible by the changes to mpy_ld that allow symbol
resolution across standard compiler-provided libraries.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit introduces an additional symbol resolution mechanism to the
natmod linking process. This allows the build scripts to look for required
symbols into selected libraries that are provided by the compiler
installation (libgcc and libm at the moment).
For example, using soft-float code in natmods, whilst technically possible,
was not an easy process and required some additional work to pull it off.
With this addition all the manual (and error-prone) operations have been
automated and folded into `tools/mpy_ld.py`.
Both newlib and picolibc toolchains are supported, albeit the latter may
require a bit of extra configuration depending on the environment the build
process runs on. Picolibc's soft-float functions aren't in libm - in fact
the shipped libm is nothing but a stub - but they are inside libc. This is
usually not a problem as these changes cater for that configuration quirk,
but on certain compilers the include paths used to find libraries in may
not be updated to take Picolibc's library directory into account. The bare
metal RISC-V compiler shipped with the CI OS image (GCC 10.2.0 on Ubuntu
22.04LTS) happens to exhibit this very problem.
To work around that for CI builds, the Picolibc libraries' path is
hardcoded in the Makefile directives used by the linker, but this can be
changed by setting the PICOLIBC_ROOT environment library when building
natmods.
Signed-off-by: Volodymyr Shymanskyy <vshymanskyi@gmail.com>
Co-authored-by: Alessandro Gatti <a.gatti@frob.it>
This allows running mpy-tool using MicroPython itself.
An appropriate test is added to CI to make sure it continues to work.
Signed-off-by: Volodymyr Shymanskyy <vshymanskyi@gmail.com>
Signed-off-by: Angus Gratton <angus@redyak.com.au>
Changing runner OS can change Python version, and ESP-IDF installs are
keyed on ESP-IDF and Python version together.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
This commit adds the natmod tests for the MPS2_AN385 board running
inside QEMU to the CI pipeline. Now natmod tests capabilities are equal
between the Arm and RV32 platforms for the QEMU port.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit brings the natmod tests in the CI build process for the RV32
platform. Not all example natmods are tested at the moment, as
`features` requires soft-float support, and `btree` needs thread-local
storage support in `mpy_ld.py` when built with the CI's toolchain.
Co-authored-by: Damien George <damien@micropython.org>
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
Similar to the ESP32 builds, but needs additional step to pass the
ccache directory through to the Zephyr container.
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
This image is 15GB so in theory this may be faster, although
in testing the improvement is either non-existent or marginal.
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
Can save several minutes downloading the Zephyr docker image and/or cloning
repo from GitHub. Cache keyed on the Zephyr version, which AFAIK is the
only determinant for the workspace contents.
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
Reasons to remove this:
- GitHub's macOS runners install this package by default nowadays.
- Brew renamed this package to 'pkgconf' so installing the old name on top
of the new package name has started failing.
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
Previously to this commit, running the test suite on a bare-metal board
required specifying the target (really platform) and device, eg:
$ ./run-tests.py --target pyboard --device /dev/ttyACM1
That's quite a lot to type, and you also need to know what the target
platform is, when a lot of the time you either don't care or it doesn't
matter.
This commit makes it easier to run the tests by replacing both of these
options with a single `--test-instance` (`-t` for short) option. That
option specifies the executable/port/device to test. Then the target
platform is automatically detected.
The `--test-instance` can be passed:
- "unix" (the default) to use the unix version of MicroPython
- "webassembly" to test the webassembly port
- anything else is considered a port/device to pass to Pyboard
There are also some shortcuts to specify a port/device, following
`mpremote`:
- a<n> is short for /dev/ttyACM<n>
- u<n> is short for /dev/ttyUSB<n>
- c<n> is short for COM<n>
For example:
$ ./run-tests.py -t a1
Note that the default test instance is "unix" and so this commit does not
change the standard way to run tests on the unix port, by just doing
`./run-tests.py`.
As part of this change, the platform (and it's native architecture if it
supports importing native .mpy files) is show at the start of the test run.
Signed-off-by: Damien George <damien@micropython.org>
Previously the code size comparison was between the merge base (i.e. where
the PR branched), and the generated merge commit into master. If the PR
branch was older than current master, this meant the size comparison could
incorrectly include changes already merged on master but missing from the
PR branch.
This commit changes it to compare the generated merge commit against
current master, i.e. the size impact if this PR was to be merged.
This commit also disables running the code size check on "push", it now
only runs on pull_request events.
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
Fixes the problem noted at
https://github.com/micropython/micropython/pull/15547#issuecomment-2434479702
which is that, because default CI HEAD for a PR is a (generated) merge
commit into the master branch's current HEAD, then if the PR branch isn't
fully rebased then the commit check runs against commits from master as
well!
Also drops running this check on push, the pull_request event is triggered
by default on open and update ("synchronized" event), which probably covers
the cases where this check should run.
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
Updates the Zephyr port build instructions and CI to use the latest
Zephyr release tag.
Tested on frdm_k64f.
Signed-off-by: Maureen Helm <maureen.helm@analog.com>
This commit adds the `VIRT_RV32` board to the list of targets for
calculating code size changes as part of the CI pipeline.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit lets the RV64 port use the version of libffi that is bundled
as a submodule in the MicroPython source tree, as the packaged libffi
library coming from Ubuntu's RISC-V repository trashes foreign function
call results on exit.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
The Unix port's Arm target CI steps have been updated to be more in
line with the other targets (the MicroPython binary doesn't need an
environment variable to be set in order to run now).
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
The FFI helper definition was accidentally omitted when committing the
necessary shell code for building RV64 Unix builds in the CI
environment.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
The Unix port's MIPS target CI steps have been updated to be more in
line with the other targets (the MicroPython binary now runs as a
dynamic executable), and the test exceptions for ffi have been lifted.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
Currently both the qemu-arm and qemu-riscv ports share a lot of code and
functionality. This commit merges the qemu-riscv port into the qemu-arm
port. The only real differences between the two are the toolchains used to
build the code, and the initialisation/startup framework. Everything else
is pretty much the same, so this brings the following benefits:
- less code duplication
- less burden on maintenance
- generalised qemu port, could in the future support other architectures
A new board `VIRT_RV32` has been added to the qemu-arm port which is the
existing RISC-V board from the qemu-riscv port. To build it:
$ make BOARD=VIRT_RV32 repl
To cleanly separate the code for the different architectures, startup code
has been moved to ports/qemu-arm/mcu/<arch>/.
Signed-off-by: Damien George <damien@micropython.org>
Currently, the qemu-arm (and qemu-riscv) port has two build modes:
- a simple test that executes a Python string; and
- a full test that uses tinytest to embed all tests within the firmware,
then executes that and captures the output.
This is very different to all the other ports. A difficulty with using
tinytest is that with the large number of tests the firmware overflows its
virtual flash size. It's also hard to run tests via .mpy files and with
the native emitter. Being different to the other ports also means an extra
burden on maintenance.
This commit reworks the qemu-arm port so that it has a single build target
that creates a standard firmware which has a REPL. When run under
qemu-system-arm, the REPL acts like any other bare-metal port, complete
with soft reset (use machine.reset() to turn it off and exit
qemu-system-arm).
This approach gives many benefits:
- allows playing with a REPL without hardware;
- allows running the test suite as it would on a bare-metal board, by
making qemu-system-arm redirect the UART serial of the virtual device to
a /dev/pts/xx file, and then running run-tests.py against that serial
device;
- skipping tests is now done via the logic in `run-tests.py` and no longer
needs multiple places to define which tests to skip
(`tools/tinytest-codegen.py`, `ports/qemu-arm/tests_profile.txt` and also
`tests/run-tests.py`);
- allows testing/using mpremote with the qemu-arm port.
Eventually the qemu-riscv port would have a similar change.
Prior to this commit the test results were:
743 tests ok. (121 skipped)
With this commit the test results are:
753 tests performed (22673 individual testcases)
753 tests passed
138 tests skipped
More tests are skipped because more are included in the run. But overall
more tests pass.
Signed-off-by: Damien George <damien@micropython.org>
With this change, the zephyr port is tested against the standard test suite
via the following scheme:
- the zephyr port is built with the `qemu_cortex_m3` board and the
`prj_minimal.conf` configuration
- `qemu-system-arm` runs `zephyr.elf`
- the zephyr console is redirected to a pts/pty
- `tests/run-tests.py` is run in bare-metal mode against the pts/pty device
This allows testing the zephyr port as though it were a physical board
attached over a serial port.
Signed-off-by: Damien George <damien@micropython.org>
