The argument corresponding to a `%q` specifier must be of type `qstr`, not
a narrower type like `int16_t`. Not ensuring this caused an assertion
error on one Windows x64 build.
The argument corresponding to a `%d` specifier must be of type `int`, not a
potentially-wider type like `mp_uint_t`. Not ensuring this prevented the
function name from being printed on the unix nanbox build.
Signed-off-by: Jeff Epler <jepler@gmail.com>
If the fields added for `MICROPY_PY_SYS_SETTRACE` are not initialized
properly, their value in a thread is indeterminate. In particular, if the
callback is not NULL, it will be invoked as a function.
Signed-off-by: Jeff Epler <jepler@gmail.com>
This commit introduces an optional feature to provide to native emitters
the fully qualified name of the entity they are compiling.
This is achieved by altering the generic ASM API to provide a third
argument to the entry function, containing the name of the entity being
compiled. Currently only the debug emitter uses this feature, as it is
not really useful for other emitters for the time being; in fact the
macros in question just strip the name away.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit cleans up the single entry point integer-indexed load/store
emitters that have been built by merging the single operand type
load/store functions in 1f5ba6998b.
To follow the same convention found in RV32 and Xtensa emitters, the
function operand size is not named after the left shift amount to apply
to the initial offset to get its true byte offset, but by a generic
"operand size".
Also, those functions were updated to use the new MP_FIT_UNSIGNED macros
to perform bit width checks when figuring out which opcode encoding is
the best one to use.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit expands the implementation of Viper load/store operations
that are optimised for the x86 platform.
Like x86, x64 already implemented all necessary functions and all it
took to expose these to Viper after the infrastructure refactoring
was to add a few defines.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit expands the implementation of Viper load/store operations
that are optimised for the x86 platform.
Unlike other platforms, x86 already implemented all necessary
functions and all it took to expose these to Viper after the
infrastructure refactoring was to add a few defines.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit expands the implementation of Viper load/store operations
that are optimised for the Xtensa platform.
Now both load and store emitters should generate the shortest possible
sequence in all cases. Redundant specialised operation emitters have
been aliased to the general case implementation - this was the case of
integer-indexed load/store operations with a fixed offset of zero.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit expands the implementation of Viper load/store operations
that are optimised for the Arm platform.
Now both load and store emitters should generate the shortest possible
sequence in all cases. Redundant specialised operation emitters have
been folded into the general case implementation - this was the case of
integer-indexed load/store operations with a fixed offset of zero.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit expands the implementation of Viper load/store operations
that are optimised for the RV32 platform.
Given the way opcodes are encoded, all value sizes are implemented with
only two functions - one for loads and one for stores. This should
reduce duplication with existing operations and should, in theory, save
space as some code is removed. Both load and store emitters will
generate the shortest possible sequence (as long as the stack pointer is
not involved), using compressed opcodes when possible.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit adds two macros that lets check whether a given value can
fit an arbitrary number of bits, either as a signed or as an unsigned
number.
The native emitter code backends perform a lot of bit size checks to see
if a particular code sequence can be emitted instead of a generic one,
and each platform backend has its own ad-hoc macros (usually one per bit
count and signedness).
With these macros there's a single way to perform those checks, plus
there's no more chance for off-by-one mask length errors when dealing
with signed numbers.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
Tested using gcc 7.3.1 which does not have the popcount built-in and uses
this fallback version. Without the fix, mpy-cross produces mpy files with
corrupt RISC-V machine code. With the fix, mpy-cross output is correct.
Signed-off-by: Damien George <damien@micropython.org>
CPython math.nan is positive with regards to copysign. The signaling bit
(aka sign flag) was incorrectly set.
In addition, REPR_C and REPR_D should only use the _true_ nan to prevent
system crash in case of hand-crafted floats. For instance, with REPR_C,
any nan-like float following the pattern
`01111111 1xxxxxxx xxxxxxxx xxxxx1xx` would be switched to an immediate
object or a qstr string. When the qstr index is too large, this would
cause a crash.
This commit fixes the issue, and adds the relevant test cases.
Signed-off-by: Yoctopuce dev <dev@yoctopuce.com>
When the symbol `__all__` is defined in a module, `mp_import_all()` should
import all listed symbols into the global environment, rather than relying
on the underscore-is-private default. This is the standard in CPython.
Each item is loaded in the same way as if it would be an explicit import
statement, and will invoke the module's `__getattr__` function if needed.
This provides a straightforward solution for fixing star import of modules
using a dynamic loader, such as `extmod/asyncio` (see issue #7266).
This improvement has been enabled at BASIC_FEATURES level, to avoid
impacting devices with limited ressources, for which star import is of
little use anyway.
Additionally, detailled reporting of errors during `__all__` import has
been implemented to match CPython, but this is only enabled when
ERROR_REPORTING is set to MICROPY_ERROR_REPORTING_DETAILED.
Signed-off-by: Yoctopuce dev <dev@yoctopuce.com>
Any '_' variables/functions in frozen modules are currently printed, when
they shouldn't be. That's due to underscore names possibly existing
between the start and end qstrs which are used to print the auto-complete
matches. The underscore names should be skipped when iterating between the
two boundary qstrs.
The underscore attributes are removed from the extra coverage exp file
because tab completing "import <tab>" no longer lists modules beginning
with an underscore.
Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
Previously, there was no test coverage of the "write failed" path. In
fact, the assertion would fire instead of gracefully raising a Python
exception.
Slightly re-organize the code to place the assertion later. Add a test
case which exercises all paths, and update the expected output.
Signed-off-by: Jeff Epler <jepler@gmail.com>
By ensuring the value to be shifted is an unsigned of the appropriate type.
This fixes several runtime diagnostics such as:
../../py/binary.c:199:28: runtime error:
left shift of 32768 by 16 places
cannot be represented in type 'int'
Signed-off-by: Jeff Epler <jepler@gmail.com>
In the case of viper code it's possible to reach MP_ASM_PASS_EMIT with a
code size of 0 bytes. Update the assertion accordingly.
After this change, `mpy-cross -march=debug' on viper tests no longer
crashes.
Fixes issue #17467.
Signed-off-by: Jeff Epler <jepler@gmail.com>
Looking at the git history, there's no indication that the
`PF_FLAG_NO_TRAILZ` flag was ever implemented or that "%!" was used as an
`mp_printf` format string in practice.
So remove the flag and re-number the other flags.
Leave `PF_FLAG_SEP_POS` at 9 (the highest position that probably works with
16-bit integers like the pic16bit port).
Signed-off-by: Jeff Epler <jepler@gmail.com>
By refactoring the code to separate out the slicing operation from the
regular indexing operation, code can be shared between the various types of
slice operations (read/assign/delete).
Signed-off-by: Jeff Epler <jepler@gmail.com>
In the case where an mpz number is zero, its `len` is 0 and its `dig` is
NULL. In that case, decrementing NULL via `d--` is undefined behavior
according to the C specification.
Restructuring the loops in this way avoids undefined behavior.
Also, ensure that these cases are tested in the coverage test. This
doesn't make much difference now, but would otherwise cause errors later
when the undefined behavior sanitizer is employed in CI.
Signed-off-by: Jeff Epler <jepler@gmail.com>
If a complex literal had a negative real part and a positive imaginary
part, it was not parsed properly because the imaginary part also came out
negative.
Includes a test of complex parsing, which fails without this fix.
Co-authored-by: ComplexSymbol <141301057+ComplexSymbol@users.noreply.github.com>
Signed-off-by: Jeff Epler <jepler@gmail.com>
As suggested by @dpgeorge, factor out part of array_construct to allow it
to be used for construction & extension.
Note that extending with a known-length list (or tuple) goes through the
slow path of calling array_extend once per element.
Fixes issue #7408.
Signed-off-by: Jeff Epler <jepler@gmail.com>
A few more bytes can be saved by not using nested `if`s (4 bytes for
`build-MICROBIT/py/parsenum.o`, 8 bytes for RPI_PICO firmware).
This commit is better viewed with whitespace changes hidden, because
two blocks were reindented (e.g., `git show -b`).
Signed-off-by: Jeff Epler <jepler@gmail.com>
By avoiding two different checks of the string length, code size is reduced
without changing behavior: Some invalid float/complex strings like "ix"
will get handled just like "xx" in the main number literal parsing code
instead.
The optimizer alone couldn't remove the reundant comparisons because it
couldn't make a transformation that let an invalid string like "ix" pass
into the generic number parsing code.
Signed-off-by: Jeff Epler <jepler@gmail.com>
With this change, all tests (except thread tests) now pass on RPI_PICO when
using the native emitter:
(plug in RPI_PICO)
$ cd tests
$ ./run-tests.py -t a0 --via-mpy --emit native
Signed-off-by: Damien George <damien@micropython.org>
This commit lets the native emitter backend extends the range of the
BCC family of opcodes (BALL, BANY, BBC, BBS, BEQ, BGE, BGEU, BLT,
BLTU, BNALL, BNE, BNONE) from 8 bits to 18 bits.
The test suite contains some test files that, when compiled into native
code, would require BCC jumps outside the (signed) 8 bits range. In
this case either the MicroPython interpreter or mpy-cross would raise an
exception, not running the test when using the "--via-mpy --emit native"
command line options with the test runner.
This comes with a 3 bytes penalty on each forward jump, bringing the
footprint of those jumps to 6 bytes each, as a longer opcode sequence
has to be emitted to let jumps access a larger range. However, this is
slightly offset by the fact that backward jumps can be emitted with a
single opcode if the range is small enough (8-bits offset).
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit lets the native emitter backend extends the range of the
BCCZ family of opcodes (BEQZ, BNEZ, BLTZ, BGEZ) from 12 bits to 18
bits.
The test suite contains some test files that, when compiled into native
code, would require BCCZ jumps outside the (signed) 12 bits range. In
this case either the MicroPython interpreter or mpy-cross would raise an
exception, not running the test when using the "--via-mpy --emit native"
command line options with the test runner.
This comes with a 3 bytes penalty on each forward jump, bringing the
footprint of those jumps to 6 bytes each, as a longer opcode sequence
has to be emitted to let jumps access a larger range. However, this is
slightly offset by the fact that backward jumps can be emitted with a
single opcode if the range is small enough (3 bytes for a 12-bits
offset).
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit performs a small refactoring on the Arm native emitter, by
renaming all but one instance of ASM_ARM_REG_R8 into REG_TEMP.
ASM_ARM_REG_R8 is the temporary register used by the emitter when
operations cannot overwrite the value of a particular register and some
extra storage is needed.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit extends the range for int-indexed load/store opcode
generators, making them emit correct code sequences for offsets that
span more than 12 bits.
This is necessary due to those generator bits being also used in the
Viper emitter, where it's more probable to reference offsets that can
not be embedded in the LDR/STR opcodes.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit removes redundant RV32 implementations of certain
int-indexed code generation operations (32-bit load/store and 16-bit
load).
Those operations were already available as part of the native emitter
API but were not exposed to the Viper code generator. As part of the
introduction of more specialised load and store API calls to
int-indexed Viper load/store generator bits, the existing native emitter
implementations are reused, thus making the Viper implementations
redundant.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit updates the existing specialised implementations for
int-indexed 32-bit load and store operations, and adds a specialised
implementation for int-indexed 16-bit load.
The 32-bit operations relied on the fact that their applicability was
limited to a specific range, falling back on a generic implementation
otherwise. Introducing a single entry point for each int-indexed
load/store operation size would break that assumption. Now those two
operations contain fallback code to generate working code by themselves
instead of raising an exception.
The 16-bit operation instead simply did not have any range check, but it
was not exposed directly to the Viper emitter. When a 16-bit
int-indexed load entry point was introduced, the existing implementation
would fail when accessing memory outside its 0..255 halfwords range. A
specialised implementation is now present, performing fewer operations
than the existing Viper emitter equivalent.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit lets the Thumb native code generator backend emit ARMv7-M
specific opcodes for indexed load/store operations if possible.
Now T3 opcode encodings are used if the generator backend is configured
to allow emitting ARMv7-M opcodes and if the (unsigned) scaled index
fits in 12 bits. Or, in other words, LDR{B,H}.W and STR{B,H}.W opcodes
are now emitted if possible.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit extends the generic ASM API by adding the rest of the
ASM_{LOAD,STORE}[size]_REG_REG_OFFSET macros whenever applicable.
The Viper int-indexed load/store code generator was changed to use those
API functions if they are available, falling back to backend-specific
implementations if possible and ultimately to a generic implementation.
Right now all backends except for x64 implement load16, load32, and
store32 operations (x64 only implements load16).
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit lets mpy_ld.py resolve symbols not only from the object
files involved in the linking process, or from compiler-supplied static
libraries, but also from a list of symbols referenced by an absolute
address (usually provided by the system's ROM).
This is needed for ESP8266 targets as some C stdlib functions are
provided by the MCU's own ROM code to reduce the final code footprint,
and therefore those functions' implementation was removed from the
compiler's support libraries. This means that unless `LINK_RUNTIME` is
set (which lets tooling look at more libraries to resolve symbols) the
build process will fail as tooling is unaware of the ROM symbols'
existence. With this change, fixed-address symbols can be exposed to
the symbol resolution step when performing natmod linking.
If there are symbols coming in from a fixed-address symbols list and
internal code or external libraries, the fixed-address symbol address
will take precedence in all cases.
Although this is - in theory - also working for the whole range of ESP32
MCUs, testing is currently limited to Xtensa processors and the example
natmods' makefiles only make use of this commit's changes for the
ESP8266 target.
Natmod builds can set the MPY_EXTERN_SYM_FILE variable pointing to a
linkerscript file containing a series of symbols (weak or strong) at a
fixed address; these symbols will then be used by the MicroPython
linker when packaging the natmod. If a different natmod build method is
used (eg. custom CMake scripts), `tools/mpy_ld.py` can now accept a
command line parameter called `--externs` (or its short variant `-e`)
that contains the path of a linkerscript file with the fixed-address
symbols to use when performing the linking process.
The linkerscript file parser can handle a very limited subset of
binutils's linkerscript syntax, namely just block comments, strong
symbols, and weak symbols. Each symbol must be in its own line for the
parser to succeed, empty lines or comment blocks are skipped. For an
example of what this parser was meant to handle, you can look at
`ports/esp8266/boards/eagle.rom.addr.v6.ld` and follow its format.
The natmod developer documentation is also updated to reflect the new
command line argument accepted by `mpy_ld.py` and the use cases for the
changes introduced by this commit.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
Without this change, a scheduler callback which itself queues a new
callback will have that callback executed as part of the same scheduler
run. Where a callback may re-queue itself, this can lead to an infinite
loop.
With this change, each call to mp_handle_pending() will only service the
callbacks which were queued when the scheduler pass started - any callbacks
added during the run are serviced on the next mp_handle_pending().
This does mean some interrupts may have higher latency (as callback is
deferred until next scheduler run), but the worst-case latency should stay
very similar.
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
This commit simplifies native functions' prologue code by not emitting a
jump opcode that goes over the function's constants pool if the pool is
empty.
The original code assumed the constants pool is never empty as large
32-bits constants are commonly used, but for inline assembler functions
that may not be the case. This meant that inline assembler functions
may start with an unneeded jump (along with its alignment byte), using
four bytes more than necessary.
This commit is limited to the "xtensa" target, as "xtensawin" doesn't
support inline assembler functions yet, so native functions' constant
pools are almost always guaranteed to hold one or more values.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit expands the Xtensa inline assembler to support most if not
all opcodes available on the ESP8266 and LX3 Xtensa cores.
This is meant as a stepping stone to add inline assembler support for
the ESP32 and its LX6 core, along to windowed-specific opcodes and
additional opcodes that are present only on the LX7 core (ESP32-S3 and
later).
New opcodes being added are covered by tests, and the provided tests
were expanded to also include opcodes available in the existing
implementation. Given that the ESP8266 space requirements are tighter
than ESP32's, certain opcodes that won't be commonly used have been put
behind a define to save some space in the general use case.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit fixes compilation errors occurring when enabling the Xtensa
code dumper inside mpy-cross.
The original code was meant to dump the code from an Xtensa device
itself, but for debugging the inline assembler this functionality was
also needed off-line. The changes involve solving a signed/unsigned
mismatch that was not much of a problem for the 8266's gcc version but
made modern compilers complain, and using the printf formatter for
pointers when it comes to printing code addresses.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit removes old raw printf calls happening inside certain branch
opcode emitters, indicating the target label is out of range for the
opcode. They have been replaced with a RuntimeError being raised in
these cases, using a parameterised qstr instead.
Whilst this technically breaks runtime behaviour expectations, the
generated code would not have worked anyway so it's better to catch
those cases early. This should be updated to always emit long jumps
unless jumps are backwards and short enough, following the other ports,
but that's something coming later.
This is actually needed because there are test files that do not work
when processed through mpy-cross and entirely converted to native code.
The original implementation would still generate mostly-valid code that
was bound to crash on the device, whilst this change would prevent
invalid code to even be emitted in the first place.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
Allows both MICROPY_PERSISTENT_CODE_TRACK_FUN_DATA and MP_PLAT_COMMIT_EXEC
to be enabled at the same time.
Signed-off-by: Damien George <damien@micropython.org>
This commit performs some minor clean up for the code involved in Viper
load/store operations when said operations have an integer index.
Most platform-specific code blocks were able to generate correct opcodes
even when the index is 0, but they would still fall back to the general
case. The general case would still emit a shortened opcode sequence so
this commit does not alter the overall behaviour, but makes it easier to
extend platform-specific code whenever the full index range is going to
be handled rather than a subset of indices as it is now.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit lets the Thumb native emitter generate a proper opcode
sequence when calculating an indexed register offset for load/store
operations with said offset beight both greater than 65535 and not
able to be represented as a shifted 8-bit bitmask.
The original code would assume the scaled index would always fit in 16
bits and silently discard upper bits of the offset. Now an optimised
constant loading sequence is emitted instead, and the final offset is
also stored in the correct register in all cases.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit removes the ASM_LOAD_REG_REG and ASM_STORE_REG_REG generic
ASM API opcodes from all backends, as they are not used anymore in the
native emitter framework.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit cleans up the Viper code generation blocks for
register-indexed load and store operations.
An attempt is made to simplify the code in the common code generator
code block, by moving architecture-specific code to the appropriate
native generation backends whenever possible. This should make that
specific bit of code in the Viper generator clearer and easier to
maintain in the long term.
To achieve this, six generic assembler meta-opcodes have been
introduced, named `ASM_{LOAD,STORE}{8,16,32}_REG_REG_REG`. A
platform-independent implementation for those operations is provided, so
backends that cannot emit a shorter sequence for the requested operation
or are fine with the platform-independent implementation can just not
provide said meta-opcodes.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit lets the Viper code generator use optimised code sequence
for register-indexed load and store operations when generating Thumb
code.
Register-indexed load and store operations for Thumb now can take at
most two machine opcodes for halfword and word values, and just a single
machine opcode for byte values. The original implementation could
generate up to four opcodes in the worst case (dealing with word
values).
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit lets the Viper code generator use optimised code sequences
for register-indexed load and store operations when generating Arm code.
The existing code defaulted to generic multi-operations code sequences
for Arm code on most cases. Now optimised implementations are provided
for register-indexed loads and stores of all data sizes, taking at most
two machine opcodes for each operation.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This groups non-decimal values by fours, such as bbb_bbbb_bbbb. It also
supports `{:_d}` to use underscore for decimal numbers (grouped in threes).
Use of incorrect ":,b" is not diagnosed.
Thanks to @dpgeorge for the suggestion to reduce code size.
Signed-off-by: Jeff Epler <jepler@gmail.com>
