Some tests are just too big for targets that don't have much heap memory,
eg `tests/extmod/vfs_rom.py`. Other tests are too large because the target
doesn't have enough IRAM for native code, eg esp8266 running
`tests/micropython/viper_args.py`.
Previously, such tests were explicitly skipped on targets known to have
little memory, eg esp8266. But this doesn't scale to multiple targets, nor
to more and more tests which are too large.
This commit addresses that by adding logic to the test runner so it can
automatically skip tests when they don't fit in the target's memory. It
does this by prepending a `print('START TEST')` to every test, and if a
`MemoryError` occurs before that line is printed then the test was too big.
This works for standard tests, tests that go via .mpy files, and tests that
run in native emitter mode via .mpy files.
For tests that are too big, it prints `lrge <test name>` on the output,
and at the end prints them on a separate line of skipped tests so they can
be distinguished. They are also distinguished in the `_result.json` file
as a skipped test with reason "too large".
Signed-off-by: Damien George <damien@micropython.org>
This commit fixes a small yet harmless issue that occurs when invoking
`ci_native_mpy_modules_build` on a persistent environment, as only X64
MPY files would be removed by the cleaning process.
Now the correct architecture is passed at all times when cleaning before
building a natmod for a particular architecture, forcing a full build of
all files to better simulate the CI environment (where there's no state
persisted between runs for this step).
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit lets the CI pipeline build all natmods for the Xtensa
target, now that ROM symbols can be used in the linking process.
The restriction was put in place due to build failures on certain
natmods for Xtensa, as ROM symbols would not be used, causing undefined
symbol errors at build time.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit adds an optional configuration option for the ESP8266 port
that, if the board rebooted due to a crash, will print to stdout some
information about the error that triggered the issue.
It is not possible using regular SDK functions to intercept errors and
print information at that stage, and the only error response from the
board is to reboot itself. This is the next best thing, print some
error information just once at boot time after the crash - the least
invasive option given the situation we're in.
This is disabled by default, and can be enabled by enabling
MICROPY_HW_HARD_FAULT_DEBUG in the port configuration - obviously with a
small increase in the firmware code footprint.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit provides the appropriate external symbol addresses to let
the "btree" example natmod build for the Xtensa platform.
On the ESP8266, unsigned integer division code isn't provided as part of
libgcc.a, libm.a, or libc.a, but it is instead provided by the ROM.
Regular builds inject the appropriate symbol addresses as part of the
linking process (see eagle.rom.addr.v6.ld), but natmods need this
information brought in from somewhere else.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit provides the appropriate external symbol addresses to let
the "deflate" example natmod build for the Xtensa platform.
Unlike other natmods that require an external symbol list to build
without bringing in the whole runtime libraries set, this natmod is
referencing the `__modsi3` symbol which was removed from the ESP8266's
SDK but not present in ROM. The latter only has a `__umodsi3`
implementation that only operates on unsigned values, and thus unable to
handle this natmod. Thus, the extended library resolution process is
enabled for this natmod as a `__modsi3` implementation is made available
that way (still using ROM symbols whenever possible). This also means
that symbols that appear in both ROM and external libraries sort of
co-exist in the final MPY file, with ROM symbols being used by natmod
code but the implementation from the library still exists in the final
MPY file, unused.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit provides the appropriate external symbol addresses to let
the "framebuf" example natmod build for the Xtensa platform.
On the ESP8266, integer division code isn't provided as part of
libgcc.a, libm.a, or libc.a, but it is instead provided by the ROM.
Regular builds inject the appropriate symbol addresses as part of the
linking process (see eagle.rom.addr.v6.ld), but natmods need this
information brought in from somewhere else.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit provides the appropriate external symbol addresses to let
the "random" example natmod build for the Xtensa platform.
On the ESP8266, signed integer division code isn't provided as part of
libgcc.a, libm.a, or libc.a, but it is instead provided by the ROM.
Regular builds inject the appropriate symbol addresses as part of the
linking process (see eagle.rom.addr.v6.ld), but natmods need this
information brought in from somewhere else.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit introduces a mechanism to customise the code that is
injected to the board when performing a native module import.
A new argument, "-b"/"--begin", is added so regular Python code can be
inserted in the injected fragment between the module file creation and
the effective module import. This is needed for running natmod tests on
ESP8266 as that board does not have enough memory to fit certain modules
unless additional configuration is performed.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit lets mpy_ld.py resolve symbols not only from the object
files involved in the linking process, or from compiler-supplied static
libraries, but also from a list of symbols referenced by an absolute
address (usually provided by the system's ROM).
This is needed for ESP8266 targets as some C stdlib functions are
provided by the MCU's own ROM code to reduce the final code footprint,
and therefore those functions' implementation was removed from the
compiler's support libraries. This means that unless `LINK_RUNTIME` is
set (which lets tooling look at more libraries to resolve symbols) the
build process will fail as tooling is unaware of the ROM symbols'
existence. With this change, fixed-address symbols can be exposed to
the symbol resolution step when performing natmod linking.
If there are symbols coming in from a fixed-address symbols list and
internal code or external libraries, the fixed-address symbol address
will take precedence in all cases.
Although this is - in theory - also working for the whole range of ESP32
MCUs, testing is currently limited to Xtensa processors and the example
natmods' makefiles only make use of this commit's changes for the
ESP8266 target.
Natmod builds can set the MPY_EXTERN_SYM_FILE variable pointing to a
linkerscript file containing a series of symbols (weak or strong) at a
fixed address; these symbols will then be used by the MicroPython
linker when packaging the natmod. If a different natmod build method is
used (eg. custom CMake scripts), `tools/mpy_ld.py` can now accept a
command line parameter called `--externs` (or its short variant `-e`)
that contains the path of a linkerscript file with the fixed-address
symbols to use when performing the linking process.
The linkerscript file parser can handle a very limited subset of
binutils's linkerscript syntax, namely just block comments, strong
symbols, and weak symbols. Each symbol must be in its own line for the
parser to succeed, empty lines or comment blocks are skipped. For an
example of what this parser was meant to handle, you can look at
`ports/esp8266/boards/eagle.rom.addr.v6.ld` and follow its format.
The natmod developer documentation is also updated to reflect the new
command line argument accepted by `mpy_ld.py` and the use cases for the
changes introduced by this commit.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
Currently, mpy-cross will crash if called as:
mpy-cross -X emit=native foo.py
because it tries to use the native emitter with no target architecture set.
Fix that by checking that an architecture is set when `-X emit=native` or
`-X emit=viper` is used.
Signed-off-by: Damien George <damien@micropython.org>
Test modified to reschedule itself based on a flag setting. Without the
change in the parent commit, this test executes the callback indefinitely
and hangs but with the change it runs only once each time
mp_handle_pending() is called.
Modified-by: Angus Gratton <angus@redyak.com.au>
Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
Without this change, a scheduler callback which itself queues a new
callback will have that callback executed as part of the same scheduler
run. Where a callback may re-queue itself, this can lead to an infinite
loop.
With this change, each call to mp_handle_pending() will only service the
callbacks which were queued when the scheduler pass started - any callbacks
added during the run are serviced on the next mp_handle_pending().
This does mean some interrupts may have higher latency (as callback is
deferred until next scheduler run), but the worst-case latency should stay
very similar.
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
Currently the `FrameBuffer.blit(buf, x, y)` method requires the `buf`
argument to be another `FrameBuffer`, which is quite restrictive because it
doesn't allow blit'ing read-only memory/data.
This commit extends `blit()` to allow the `buf` argument to be a tuple or
list of the form:
(buffer, width, height, format[, stride])
where `buffer` can be anything with the buffer protocol and may be
read-only, eg `bytes`.
Also, the palette argument to `blit()` may be of the same form.
The form of this tuple/list was chosen to be the same as the signature of
the `FrameBuffer` constructor (that saves quite a bit of code size doing it
that way).
Signed-off-by: Damien George <damien@micropython.org>
If the BLE radio stops responding before deinit is called the function can
get stuck waiting for an event that is never received, particularly if the
radio is external or on a separate core.
This commit adds a timeout, similar to the timeout already used in the init
function. Updated for nimble, btstack, esp32 and zephyr bindings.
Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
The vendor and product fields in the `board.json` files were somewhat
inconsistent. Remove any duplication of the vendor name in the product
field so that `f"{vendor} {product}"` reads well.
In addition to that, update most of the URL's for `board.json` files that
are modified here, and match case and spacing used by the manufacturers for
the vendor and product names.
Signed-off-by: Matt Trentini <matt.trentini@gmail.com>
Implementation added for various ports in the parent commits.
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
Implements MSG_PEEK and MSG_DONTWAIT (both passed through to LWIP
sockets API).
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
The `_results.json` output of `run-tests.py` was recently changed in
7a55cb6b36 to add a list of passed and
skipped tests.
The way this was done turned out to be not general enough, because we want
to add another type of result, namely tests that are skipped because they
are too large.
Instead of having separate lists in `_results.json` for each kind of result
(pass, fail, skip, skip too large, etc), this commit changes the output
form of `_results.json` so that it stores a single list of 3-tuples of all
tests that were run:
[(test_name, result, reason), ...]
That's more general and allows adding a reason for skipped and failed
tests. At the moment this reason is just an empty string, but can be
improved in the future.
Signed-off-by: Damien George <damien@micropython.org>
If the USB serial device locks up, then writes to that device can hang
forever. That can make the test runner `tests/run-tests.py` lock up, among
other problems.
This commit introduces a 5 second write-timeout, and catches any OSError's
raised during `enter_raw_repl()`. Now, if a USB serial device locks up,
`enter_raw_repl()` will eventually raise an exception.
Signed-off-by: Damien George <damien@micropython.org>
This applies the mpremote commit 03fe9c55ea
to pyboard.py.
The `timeout_overall` is used in `enter_raw_repl()`. It prevents waiting
forever for a serial device that does not respond to the Ctrl-C/Ctrl-D/etc
commands and is constantly outputting data.
Signed-off-by: Damien George <damien@micropython.org>
This applies the mpremote commit 0d46e45a1f
to pyboard.py.
If the target does not return any data then `read_until()` will block
indefinitely. Fix this by making the initial read part of the general read
look, which always checks `inWaiting() > 0` before reading from the serial
device.
Also added the UART timeout to the constructor. This is not currently used
but may be used as an additional safeguard.
Signed-off-by: Damien George <damien@micropython.org>
This commit simplifies native functions' prologue code by not emitting a
jump opcode that goes over the function's constants pool if the pool is
empty.
The original code assumed the constants pool is never empty as large
32-bits constants are commonly used, but for inline assembler functions
that may not be the case. This meant that inline assembler functions
may start with an unneeded jump (along with its alignment byte), using
four bytes more than necessary.
This commit is limited to the "xtensa" target, as "xtensawin" doesn't
support inline assembler functions yet, so native functions' constant
pools are almost always guaranteed to hold one or more values.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit expands the Xtensa inline assembler to support most if not
all opcodes available on the ESP8266 and LX3 Xtensa cores.
This is meant as a stepping stone to add inline assembler support for
the ESP32 and its LX6 core, along to windowed-specific opcodes and
additional opcodes that are present only on the LX7 core (ESP32-S3 and
later).
New opcodes being added are covered by tests, and the provided tests
were expanded to also include opcodes available in the existing
implementation. Given that the ESP8266 space requirements are tighter
than ESP32's, certain opcodes that won't be commonly used have been put
behind a define to save some space in the general use case.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit fixes compilation errors occurring when enabling the Xtensa
code dumper inside mpy-cross.
The original code was meant to dump the code from an Xtensa device
itself, but for debugging the inline assembler this functionality was
also needed off-line. The changes involve solving a signed/unsigned
mismatch that was not much of a problem for the 8266's gcc version but
made modern compilers complain, and using the printf formatter for
pointers when it comes to printing code addresses.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit removes old raw printf calls happening inside certain branch
opcode emitters, indicating the target label is out of range for the
opcode. They have been replaced with a RuntimeError being raised in
these cases, using a parameterised qstr instead.
Whilst this technically breaks runtime behaviour expectations, the
generated code would not have worked anyway so it's better to catch
those cases early. This should be updated to always emit long jumps
unless jumps are backwards and short enough, following the other ports,
but that's something coming later.
This is actually needed because there are test files that do not work
when processed through mpy-cross and entirely converted to native code.
The original implementation would still generate mostly-valid code that
was bound to crash on the device, whilst this change would prevent
invalid code to even be emitted in the first place.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
By showing the argument and refer to epoch instead of a fixed date. The
note about epoch lists the ports using the POSIX epoch.
Signed-off-by: robert-hh <robert@hammelrath.com>
Add support for the boards:
- SparkFun SAMD21 Dev Breakout
- SparkFun RedBoard Turbo
Both boards are SAMD21 based and actively sold by SparkFun.
Signed-off-by: robert-hh <robert@hammelrath.com>
Adding a QSPI memory chip on a STM32G4 does not work due to some small
issues, which are fixed in this commit:
- Rename QUADSPI1_xxx alt-func names to QUADSPI_xxx, to match the static
names used in `qspi.c`.
- Enable `mpu.h` macros on G4.
- Don't include I- and D-cache invalidation on G4.
Signed-off-by: Damien George <damien@micropython.org>
This reverts commit 62e0fa04a7.
Reverting as the only linker wrap needed for nrf port was removed
in the parent commit.
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
Previously MicroPython ports would linker-wrap dcd_event_handler
in order to schedule the USB task callback to run when needed.
TinyUSB 0.16 added proper support for an event hook to do the
same thing without the hacky linker wrapping.
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
The existing test for `math.e` and `math.pi` constants can fail on certain
targets if the functions `math.exp()` and/or `math.cos()` are not accurate
enough (eg out by an LSB of float precision). For example this test
currently fails on PYBD_SF6 which uses double precision floats (and that's
due to the `lib/libm_dbl/exp.c` implementation not being exact).
This commit changes this constant test so that it tests the actual constant
value, not the evaluation of `exp()` and `cos()` functions.
Signed-off-by: Damien George <damien@micropython.org>
Changes are:
- Refactor the open-drain macros, add GPIO_ENABLE/DISABLE_OPEN_DRAIN, and
move them to `mphalport.h`.
- Only use `uint64_t` for the open-drain mask if there are more than 32
GPIOs (saves code size).
- Ensure we're shifting a `uint64_t` by using 1ULL constants.
Signed-off-by: Phil Howard <github@gadgetoid.com>
Replace custom macros with Pico SDK functions, enabling support for RP2350B
variant chips with > 32 GPIOs.
Fixes issue #17241.
Signed-off-by: Phil Howard <github@gadgetoid.com>
Prevents lightsleep being woken up every 64ms to service LWIP timers, when:
1. No netif is up, and
2. No TCP sockets are active
The TCP socket check may not be strictly necessary, but without ticking the
tcp timer they won't ever time out by themselves.
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
