With this change, all tests (except thread tests) now pass on RPI_PICO when
using the native emitter:
(plug in RPI_PICO)
$ cd tests
$ ./run-tests.py -t a0 --via-mpy --emit native
Signed-off-by: Damien George <damien@micropython.org>
This commit lets the native emitter backend extends the range of the
BCC family of opcodes (BALL, BANY, BBC, BBS, BEQ, BGE, BGEU, BLT,
BLTU, BNALL, BNE, BNONE) from 8 bits to 18 bits.
The test suite contains some test files that, when compiled into native
code, would require BCC jumps outside the (signed) 8 bits range. In
this case either the MicroPython interpreter or mpy-cross would raise an
exception, not running the test when using the "--via-mpy --emit native"
command line options with the test runner.
This comes with a 3 bytes penalty on each forward jump, bringing the
footprint of those jumps to 6 bytes each, as a longer opcode sequence
has to be emitted to let jumps access a larger range. However, this is
slightly offset by the fact that backward jumps can be emitted with a
single opcode if the range is small enough (8-bits offset).
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit lets the native emitter backend extends the range of the
BCCZ family of opcodes (BEQZ, BNEZ, BLTZ, BGEZ) from 12 bits to 18
bits.
The test suite contains some test files that, when compiled into native
code, would require BCCZ jumps outside the (signed) 12 bits range. In
this case either the MicroPython interpreter or mpy-cross would raise an
exception, not running the test when using the "--via-mpy --emit native"
command line options with the test runner.
This comes with a 3 bytes penalty on each forward jump, bringing the
footprint of those jumps to 6 bytes each, as a longer opcode sequence
has to be emitted to let jumps access a larger range. However, this is
slightly offset by the fact that backward jumps can be emitted with a
single opcode if the range is small enough (3 bytes for a 12-bits
offset).
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit performs a small refactoring on the Arm native emitter, by
renaming all but one instance of ASM_ARM_REG_R8 into REG_TEMP.
ASM_ARM_REG_R8 is the temporary register used by the emitter when
operations cannot overwrite the value of a particular register and some
extra storage is needed.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit extends the range for int-indexed load/store opcode
generators, making them emit correct code sequences for offsets that
span more than 12 bits.
This is necessary due to those generator bits being also used in the
Viper emitter, where it's more probable to reference offsets that can
not be embedded in the LDR/STR opcodes.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit removes redundant RV32 implementations of certain
int-indexed code generation operations (32-bit load/store and 16-bit
load).
Those operations were already available as part of the native emitter
API but were not exposed to the Viper code generator. As part of the
introduction of more specialised load and store API calls to
int-indexed Viper load/store generator bits, the existing native emitter
implementations are reused, thus making the Viper implementations
redundant.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit updates the existing specialised implementations for
int-indexed 32-bit load and store operations, and adds a specialised
implementation for int-indexed 16-bit load.
The 32-bit operations relied on the fact that their applicability was
limited to a specific range, falling back on a generic implementation
otherwise. Introducing a single entry point for each int-indexed
load/store operation size would break that assumption. Now those two
operations contain fallback code to generate working code by themselves
instead of raising an exception.
The 16-bit operation instead simply did not have any range check, but it
was not exposed directly to the Viper emitter. When a 16-bit
int-indexed load entry point was introduced, the existing implementation
would fail when accessing memory outside its 0..255 halfwords range. A
specialised implementation is now present, performing fewer operations
than the existing Viper emitter equivalent.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit lets the Thumb native code generator backend emit ARMv7-M
specific opcodes for indexed load/store operations if possible.
Now T3 opcode encodings are used if the generator backend is configured
to allow emitting ARMv7-M opcodes and if the (unsigned) scaled index
fits in 12 bits. Or, in other words, LDR{B,H}.W and STR{B,H}.W opcodes
are now emitted if possible.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit extends the generic ASM API by adding the rest of the
ASM_{LOAD,STORE}[size]_REG_REG_OFFSET macros whenever applicable.
The Viper int-indexed load/store code generator was changed to use those
API functions if they are available, falling back to backend-specific
implementations if possible and ultimately to a generic implementation.
Right now all backends except for x64 implement load16, load32, and
store32 operations (x64 only implements load16).
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit lets mpy_ld.py resolve symbols not only from the object
files involved in the linking process, or from compiler-supplied static
libraries, but also from a list of symbols referenced by an absolute
address (usually provided by the system's ROM).
This is needed for ESP8266 targets as some C stdlib functions are
provided by the MCU's own ROM code to reduce the final code footprint,
and therefore those functions' implementation was removed from the
compiler's support libraries. This means that unless `LINK_RUNTIME` is
set (which lets tooling look at more libraries to resolve symbols) the
build process will fail as tooling is unaware of the ROM symbols'
existence. With this change, fixed-address symbols can be exposed to
the symbol resolution step when performing natmod linking.
If there are symbols coming in from a fixed-address symbols list and
internal code or external libraries, the fixed-address symbol address
will take precedence in all cases.
Although this is - in theory - also working for the whole range of ESP32
MCUs, testing is currently limited to Xtensa processors and the example
natmods' makefiles only make use of this commit's changes for the
ESP8266 target.
Natmod builds can set the MPY_EXTERN_SYM_FILE variable pointing to a
linkerscript file containing a series of symbols (weak or strong) at a
fixed address; these symbols will then be used by the MicroPython
linker when packaging the natmod. If a different natmod build method is
used (eg. custom CMake scripts), `tools/mpy_ld.py` can now accept a
command line parameter called `--externs` (or its short variant `-e`)
that contains the path of a linkerscript file with the fixed-address
symbols to use when performing the linking process.
The linkerscript file parser can handle a very limited subset of
binutils's linkerscript syntax, namely just block comments, strong
symbols, and weak symbols. Each symbol must be in its own line for the
parser to succeed, empty lines or comment blocks are skipped. For an
example of what this parser was meant to handle, you can look at
`ports/esp8266/boards/eagle.rom.addr.v6.ld` and follow its format.
The natmod developer documentation is also updated to reflect the new
command line argument accepted by `mpy_ld.py` and the use cases for the
changes introduced by this commit.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
Without this change, a scheduler callback which itself queues a new
callback will have that callback executed as part of the same scheduler
run. Where a callback may re-queue itself, this can lead to an infinite
loop.
With this change, each call to mp_handle_pending() will only service the
callbacks which were queued when the scheduler pass started - any callbacks
added during the run are serviced on the next mp_handle_pending().
This does mean some interrupts may have higher latency (as callback is
deferred until next scheduler run), but the worst-case latency should stay
very similar.
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
This commit simplifies native functions' prologue code by not emitting a
jump opcode that goes over the function's constants pool if the pool is
empty.
The original code assumed the constants pool is never empty as large
32-bits constants are commonly used, but for inline assembler functions
that may not be the case. This meant that inline assembler functions
may start with an unneeded jump (along with its alignment byte), using
four bytes more than necessary.
This commit is limited to the "xtensa" target, as "xtensawin" doesn't
support inline assembler functions yet, so native functions' constant
pools are almost always guaranteed to hold one or more values.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit expands the Xtensa inline assembler to support most if not
all opcodes available on the ESP8266 and LX3 Xtensa cores.
This is meant as a stepping stone to add inline assembler support for
the ESP32 and its LX6 core, along to windowed-specific opcodes and
additional opcodes that are present only on the LX7 core (ESP32-S3 and
later).
New opcodes being added are covered by tests, and the provided tests
were expanded to also include opcodes available in the existing
implementation. Given that the ESP8266 space requirements are tighter
than ESP32's, certain opcodes that won't be commonly used have been put
behind a define to save some space in the general use case.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit fixes compilation errors occurring when enabling the Xtensa
code dumper inside mpy-cross.
The original code was meant to dump the code from an Xtensa device
itself, but for debugging the inline assembler this functionality was
also needed off-line. The changes involve solving a signed/unsigned
mismatch that was not much of a problem for the 8266's gcc version but
made modern compilers complain, and using the printf formatter for
pointers when it comes to printing code addresses.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit removes old raw printf calls happening inside certain branch
opcode emitters, indicating the target label is out of range for the
opcode. They have been replaced with a RuntimeError being raised in
these cases, using a parameterised qstr instead.
Whilst this technically breaks runtime behaviour expectations, the
generated code would not have worked anyway so it's better to catch
those cases early. This should be updated to always emit long jumps
unless jumps are backwards and short enough, following the other ports,
but that's something coming later.
This is actually needed because there are test files that do not work
when processed through mpy-cross and entirely converted to native code.
The original implementation would still generate mostly-valid code that
was bound to crash on the device, whilst this change would prevent
invalid code to even be emitted in the first place.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
Allows both MICROPY_PERSISTENT_CODE_TRACK_FUN_DATA and MP_PLAT_COMMIT_EXEC
to be enabled at the same time.
Signed-off-by: Damien George <damien@micropython.org>
This commit performs some minor clean up for the code involved in Viper
load/store operations when said operations have an integer index.
Most platform-specific code blocks were able to generate correct opcodes
even when the index is 0, but they would still fall back to the general
case. The general case would still emit a shortened opcode sequence so
this commit does not alter the overall behaviour, but makes it easier to
extend platform-specific code whenever the full index range is going to
be handled rather than a subset of indices as it is now.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit lets the Thumb native emitter generate a proper opcode
sequence when calculating an indexed register offset for load/store
operations with said offset beight both greater than 65535 and not
able to be represented as a shifted 8-bit bitmask.
The original code would assume the scaled index would always fit in 16
bits and silently discard upper bits of the offset. Now an optimised
constant loading sequence is emitted instead, and the final offset is
also stored in the correct register in all cases.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit removes the ASM_LOAD_REG_REG and ASM_STORE_REG_REG generic
ASM API opcodes from all backends, as they are not used anymore in the
native emitter framework.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit cleans up the Viper code generation blocks for
register-indexed load and store operations.
An attempt is made to simplify the code in the common code generator
code block, by moving architecture-specific code to the appropriate
native generation backends whenever possible. This should make that
specific bit of code in the Viper generator clearer and easier to
maintain in the long term.
To achieve this, six generic assembler meta-opcodes have been
introduced, named `ASM_{LOAD,STORE}{8,16,32}_REG_REG_REG`. A
platform-independent implementation for those operations is provided, so
backends that cannot emit a shorter sequence for the requested operation
or are fine with the platform-independent implementation can just not
provide said meta-opcodes.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit lets the Viper code generator use optimised code sequence
for register-indexed load and store operations when generating Thumb
code.
Register-indexed load and store operations for Thumb now can take at
most two machine opcodes for halfword and word values, and just a single
machine opcode for byte values. The original implementation could
generate up to four opcodes in the worst case (dealing with word
values).
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit lets the Viper code generator use optimised code sequences
for register-indexed load and store operations when generating Arm code.
The existing code defaulted to generic multi-operations code sequences
for Arm code on most cases. Now optimised implementations are provided
for register-indexed loads and stores of all data sizes, taking at most
two machine opcodes for each operation.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This groups non-decimal values by fours, such as bbb_bbbb_bbbb. It also
supports `{:_d}` to use underscore for decimal numbers (grouped in threes).
Use of incorrect ":,b" is not diagnosed.
Thanks to @dpgeorge for the suggestion to reduce code size.
Signed-off-by: Jeff Epler <jepler@gmail.com>
This commit renames the NORETURN macro, indicating to the compiler
that a function does not return, into MP_NORETURN to maintain the same
naming convention of other similar macros.
To maintain compaitiblity with existing code NORETURN is aliased to
MP_NORETURN, but it is also deprecated for MicroPython v2.
This changeset was created using a similar process to
decf8e6a8b ("all: Remove the "STATIC"
macro and just use "static" instead."), with no documentation or python
scripts to change to reflect the new macro name.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
Avoids the new Wunterminated-string-literal when compiled with gcc 15.1.
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
IOBase is quite an important building block of other parts of the system,
such as `mpremote mount` and running .mpy and native tests.
This feature costs +244 bytes of firmware size on ARM Thumb2 architectures,
which is worth the cost for the extra features it enables.
The change here means that `io.IOBase` is now enabled on all nrf boards,
(previously it was only nRF52840 and nRF9160) and also B_L072Z_LRWAN1
(there is no change to other ports or boards).
Signed-off-by: Damien George <damien@micropython.org>
This ensures the check in MP_NLR_JUMP_HEAD works as expected and
nlr_jump_fail gets called so we get a bit better error message.
Signed-off-by: Daniël van de Giessen <daniel@dvdgiessen.nl>
Fixes thread safety issue that could cause memory corruption on ports
with (MICROPY_PY_THREAD && !MICROPY_PY_THREAD_GIL) - currently only rp2 and
unix have this configuration.
Adds unit test for TLS sockets that exercises this code path. I wasn't
able to make this fail on rp2, the race condition window is pretty narrow
and may not have a direct impact on a quiet system.
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
This requires explicitly naming and initializing all members so add that
where needed and possible. For MP_DEFINE_NLR_JUMP_CALLBACK_FUNCTION_1
this would require initializing the .callback member, but that's a bit
of a waste since the macro is always followed by a call to
nlr_push_jump_callback() to initialize exactly that member, so rewrite
the macro without initializers.
Signed-off-by: stijn <stijn@ignitron.net>
This fix handles attrtuple as well, eg. os.uname(). A test case has been
added in basics/attrtuple2.py.
Fixes issue #16969.
Signed-off-by: Yoctopuce dev <dev@yoctopuce.com>
The "index fixing" behavior of get_fast_slice_indexes are not desired here;
the underlying behavior of mp_obj_slice_indexes actually is.
Fixes issue #17016.
Signed-off-by: Jeff Epler <jepler@gmail.com>
Otherwise, when compiling on 16-bit systems (where `mp_uint_t` is 16 bits
wide) the compiler warns about "left shift count >= width of type", from
the static inline functions that have RV32_ENCODE_TYPE_xxx macros which
do a lot of bit shifting.
Signed-off-by: Damien George <damien@micropython.org>
Rather than having Make calling CMake to generate a list of submodules and
then run a Make target (which is complex and prone to masking other
errors), implement the submodule update logic in CMake itself.
Internal CMake-side changes are that GIT_SUBMODULES is now a CMake list,
and the trigger variable name is changed from ECHO_SUBMODULES to
UPDATE_SUBMODULES.
The run is otherwise 100% a normal CMake run now, so most of the other
special casing can be removed.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
This is a pretty fundamental built-in and having CPython-compatible
behaviour is beneficial. The code size increase is not much, and
ports/boards can still disable it if needed to save space.
Addresses issue #5384.
Signed-off-by: Damien George <damien@micropython.org>
This commit introduces an additional symbol resolution mechanism to the
natmod linking process. This allows the build scripts to look for required
symbols into selected libraries that are provided by the compiler
installation (libgcc and libm at the moment).
For example, using soft-float code in natmods, whilst technically possible,
was not an easy process and required some additional work to pull it off.
With this addition all the manual (and error-prone) operations have been
automated and folded into `tools/mpy_ld.py`.
Both newlib and picolibc toolchains are supported, albeit the latter may
require a bit of extra configuration depending on the environment the build
process runs on. Picolibc's soft-float functions aren't in libm - in fact
the shipped libm is nothing but a stub - but they are inside libc. This is
usually not a problem as these changes cater for that configuration quirk,
but on certain compilers the include paths used to find libraries in may
not be updated to take Picolibc's library directory into account. The bare
metal RISC-V compiler shipped with the CI OS image (GCC 10.2.0 on Ubuntu
22.04LTS) happens to exhibit this very problem.
To work around that for CI builds, the Picolibc libraries' path is
hardcoded in the Makefile directives used by the linker, but this can be
changed by setting the PICOLIBC_ROOT environment library when building
natmods.
Signed-off-by: Volodymyr Shymanskyy <vshymanskyi@gmail.com>
Co-authored-by: Alessandro Gatti <a.gatti@frob.it>
This function will attempt to create a `VfsRom` instance and mount it at
location "/rom" in the filesystem.
Signed-off-by: Damien George <damien@micropython.org>
This is a generic interface to allow querying and modifying the read-only
memory area of a device, if it has such an area.
Signed-off-by: Damien George <damien@micropython.org>
For a given MicroPython firmware/executable it can be sometimes important
to know how it was built, which variant/board configuration it came from.
This commit adds a new field `sys.implementation._build` that can help
identify the configuration that MicroPython was built with.
For now it's either:
* <VARIANT> for unix, webassembly and windows ports
* <BOARD>-<VARIANT> for microcontroller ports (the variant is optional)
In the future additional elements may be added to this string, separated by
a hyphen.
Resolves issue #16498.
Signed-off-by: Damien George <damien@micropython.org>
This change allows tuples to be passed as the prefix/suffix argument to the
`str.startswith()` and `str.endswith()` methods. The methods will return
`True` if the string starts/ends with any of the prefixes/suffixes in the
tuple.
Also adds full support for the `start` and `end` arguments to both methods
for compatibility with CPython.
Tests have been updated for the new behaviour.
Signed-off-by: Glenn Moloney <glenn.moloney@gmail.com>
The mantissa parsing code uses a floating point variable to accumulate
digits. Using an `mp_float_uint_t` variable instead and casting to
`mp_float_t` at the very end reduces code size. In some cases, it also
improves the rounding behaviour as extra digits are taken into account
by the int-to-float conversion code.
An extra test case handles the special case where mantissa overflow occurs
while processing deferred trailing zeros.
Signed-off-by: Yoctopuce dev <dev@yoctopuce.com>
