The recently merged 5e9189d6d1 now allows
temporary slices to be allocated on the C stack, which is much better than
allocating them on the GC heap.
Unfortunately there are cases where the C-allocated slice can escape and be
retained as an object, which leads to crashes (because that object points
to the C stack which now has other values on it).
The fix here is to add a new `MP_TYPE_FLAG_SUBSCR_ALLOWS_STACK_SLICE`.
Native types should set this flag if their subscr method is guaranteed not
to hold on to a reference of the slice object.
Fixes issue #17733 (see also #17723).
Signed-off-by: Damien George <damien@micropython.org>
Current implementation of REPR_C works by clearing the two lower bits of
the mantissa to zero. As this happens after each floating point operation,
this tends to bias floating point numbers towards zero, causing decimals
like .9997 instead of rounded numbers. This is visible in test cases
involving repeated computations, such as `tests/misc/rge_sm.py` for
instance.
The suggested fix fills in the missing bits by copying the previous two
bits. Although this cannot recreate missing information, it fixes the bias
by inserting plausible values for the lost bits, at a relatively low cost.
Some float tests involving irrational numbers have to be softened in case
of REPR_C, as the 30 bits are not always enough to fulfill the expectations
of the original test, and the change may randomly affect the last digits.
Such cases have been made explicit by testing for REPR_C or by adding a
clear comment.
The perf_test fft code was also missing a call to round() before casting a
log_2 operation to int, which was causing a failure due to a last-decimal
change.
Signed-off-by: Yoctopuce dev <dev@yoctopuce.com>
This commit provides helpers to retrieve integer values from
mp_obj_t when the content does not fit in a 32 bits integer,
without risking an implicit wrap due to an int overflow.
Signed-off-by: Yoctopuce dev <dev@yoctopuce.com>
CPython math.nan is positive with regards to copysign. The signaling bit
(aka sign flag) was incorrectly set.
In addition, REPR_C and REPR_D should only use the _true_ nan to prevent
system crash in case of hand-crafted floats. For instance, with REPR_C,
any nan-like float following the pattern
`01111111 1xxxxxxx xxxxxxxx xxxxx1xx` would be switched to an immediate
object or a qstr string. When the qstr index is too large, this would
cause a crash.
This commit fixes the issue, and adds the relevant test cases.
Signed-off-by: Yoctopuce dev <dev@yoctopuce.com>
This requires explicitly naming and initializing all members so add that
where needed and possible. For MP_DEFINE_NLR_JUMP_CALLBACK_FUNCTION_1
this would require initializing the .callback member, but that's a bit
of a waste since the macro is always followed by a call to
nlr_push_jump_callback() to initialize exactly that member, so rewrite
the macro without initializers.
Signed-off-by: stijn <stijn@ignitron.net>
The `mp_obj_code_t` and `mp_type_code` code object was defined internally
in both `py/builtinevex.c` and `py/profile.c`, with completely different
implementations (the former very minimal, the latter quite complete).
This commit factors these implementations into a new, separate source file,
and allows the code object to have four different modes, selected at
compile-time:
- MICROPY_PY_BUILTINS_CODE_NONE: code object not included in the build.
- MICROPY_PY_BUILTINS_CODE_MINIMUM: very simple code object that just holds
a reference to the function that it represents. This level is used when
MICROPY_PY_BUILTINS_COMPILE is enabled.
- MICROPY_PY_BUILTINS_CODE_BASIC: simple code object that holds a reference
to the proto-function and its constants.
- MICROPY_PY_BUILTINS_CODE_FULL: almost complete implementation of the code
object. This level is used when MICROPY_PY_SYS_SETTRACE is enabled.
Signed-off-by: Damien George <damien@micropython.org>
Allowing passing keyword arguments to a native base's __init__, i.e.
`make_new` in the C code. Previously only positional arguments were
allowed.
The main trade-off in this commit is that every call to the native base's
`make_new` is now going to be preceded by a call to
`mp_map_init_fixed_table` even though most of what that does is unused and
instead it merely serves as a way to pass the number of keyword arguments.
Fixes issue #15465.
Signed-off-by: stijn <stijn@ignitron.net>
This commit adds a new `RingIO` type which exposes the internal ring-buffer
code for general use in Python programs. It has the stream interface
making it similar to `StringIO` and `BytesIO`, except `RingIO` has a fixed
buffer size and is automatically safe when reads and writes are in
different threads or an IRQ.
This new type is enabled at the "extra features" ROM level.
Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
These were changed in v1.11 (2019). Prepare to remove the compatibility
macros as part of V2 changes.
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
Also put this function inside the `MICROPY_PY_BUILTINS_SLICE` guard,
because it's only usable when that option is enabled.
Signed-off-by: Damien George <damien@micropython.org>
There were lots of places where this pattern was duplicated, to convert a
standard C string to a MicroPython string:
x = mp_obj_new_str(s, strlen(s));
This commit provides a simpler method that removes this code duplication:
x = mp_obj_new_str_from_cstr(s);
This gives clearer, and probably smaller, code.
Signed-off-by: Jon Foster <jon@jon-foster.co.uk>
This commit swaps the order of the `flags` and `name` struct initialisers
for `mp_obj_type_t`, to fix an incompatibility with C++. The original
order of the initialiser didn't match the definition of the type, and
although that's still legal C, it's not legal C++.
Signed-off-by: Vonasmic <kasarkal123@gmail.com>
Viper functions are quite different to native functions and benefit from
being a separate type. For example, viper functions don't have a bytecode-
style prelude, and don't support generators or default arguments.
Signed-off-by: Damien George <damien@micropython.org>
Following 709e8328d9.
Using this helps to reduce code size. And it ensure that the type is
always set as soon as the object is allocated, which is important for the
GC to function correctly.
Signed-off-by: Damien George <damien@micropython.org>
This allows mp_get_buffer_raise() to be changed to a simple inline function
that in the majority of cases costs the same (in code size) to call as the
original mp_get_buffer_raise(), because the flags argument is a constant.
Signed-off-by: Damien George <damien@micropython.org>
This behaviour matches CPython. It's useful to be able to store bound
method objects in dicts/sets, and compare for equality, eg when storing
them in a list and using list.remove().
Signed-off-by: Daniël van de Giessen <daniel@dvdgiessen.nl>
This replaces the previous QSTR_null entry in the globals dict which could
leak out to Python (e.g. via iteration of mod.__dict__) and could lead to
crashes.
It results in smaller code size at the expense of turning a lookup into a
loop, but the list it is looping over likely only contains one or two
elements.
To allow a module to register its custom attr function it can use the new
`MP_REGISTER_MODULE_DELEGATION` macro.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This makes it so that sub-packages are resolved relative to their parent's
`__path__`, rather than re-resolving each parent's filesystem path.
The previous behavior was that `import foo.bar` would first re-search
`sys.path` for `foo`, then use the resulting path to find `bar`.
For already-loaded and u-prefixed modules, because we no longer need to
build the path from level to level, we no longer unnecessarily search
the filesystem. This should improve startup time.
Explicitly makes the resolving process clear:
- Loaded modules are returned immediately without touching the filesystem.
- Exact-match of builtins are also returned immediately.
- Then the filesystem search happens.
- If that fails, then the weak-link handling is applied.
This maintains the existing behavior: if a user writes `import time` they
will get time.py if it exits, otherwise the built-in utime. Whereas `import
utime` will always return the built-in.
This also fixes a regression from a7fa18c203
where we search the filesystem for built-ins. It is now only possible to
override u-prefixed builtins. This will remove a lot of filesystem stats
at startup, as micropython-specific modules (e.g. `pyb`) will no longer
attempt to look at the filesystem.
Added several improvements to the comments and some minor renaming and
refactoring to make it clearer how the import mechanism works. Overall
code size diff is +56 bytes on STM32.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Since converting to variable sized slots in mp_obj_type_t, we can now
reduce the code size a bit by removing mp_generic_unary_op() and the
corresponding slots where it is used. Instead we just implement the
generic `__hash__` operation in the runtime.
Signed-off-by: David Lechner <david@pybricks.com>
Unless MICROPY_OBJ_REPR == MICROPY_OBJ_REPR_D, these macros only work with
values and "->"/"." expressions as their sole argument. In other words,
the macros are broken with expressions which contain operations of lower
precedence than the cast operator.
Depending on situation, the old code either results in compiler error:
MP_OBJ_TO_PTR(flag ? o1 : o2) expands into "(void *)flag ? o1 : o2",
which some compiler configurations will reject (e.g. GCC -Wint-conversion
-Wint-to-pointer-cast -Werror)
Or in an incorrect address calculation:
For ptr declared as "uint8_t *" the MP_OBJ_FROM_PTR(ptr + off)
expands into ((mp_obj_t)ptr) + off, resulting in an obviously
wrong address.
Signed-off-by: Alex Riesen <alexander.riesen@cetitec.com>
This was previously used for the definition of NIC types, but they have
been updated to use a protocol instead.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Prevents double-precision floats being enabled on 32-bit architectures
where they will not fit into the mp_obj_t encoding.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
The check for make_new (i.e. used to determine something's type) is now
more complicated due to the slot access. This commit changes the inlining
of a few frequently-used helpers to overall improve code size and
performance.
Instead of being an explicit field, it's now a slot like all the other
methods.
This is a marginal code size improvement because most types have a make_new
(100/138 on PYBV11), however it improves consistency in how types are
declared, removing the special case for make_new.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
The goal here is to remove a slot (making way to turn make_new into a slot)
as well as reduce code size by the ~40 references to mp_identity_getiter
and mp_stream_unbuffered_iter.
This introduces two new type flags:
- MP_TYPE_FLAG_ITER_IS_ITERNEXT: This means that the "iter" slot in the
type is "iternext", and should use the identity getiter.
- MP_TYPE_FLAG_ITER_IS_CUSTOM: This means that the "iter" slot is a pointer
to a mp_getiter_iternext_custom_t instance, which then defines both
getiter and iternext.
And a third flag that is the OR of both, MP_TYPE_FLAG_ITER_IS_STREAM: This
means that the type should use the identity getiter, and
mp_stream_unbuffered_iter as iternext.
Finally, MP_TYPE_FLAG_ITER_IS_GETITER is defined as a no-op flag to give
the default case where "iter" is "getiter".
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
The existings mp_obj_type_t uses a sparse representation for slots for the
capability methods of the type (eg print, make_new). This commit adds a
compact slot-index representation. The basic idea is that where the
mp_obj_type_t struct used to have 12 pointer fields, it now has 12 uint8_t
indices, and a variable-length array of pointers. So in the best case (no
fields used) it saves 12x4-12=36 bytes (on a 32-bit machine) and in the
common case (three fields used) it saves 9x4-12=24 bytes.
Overall with all associated changes, this slot-index representation reduces
code size by 1000 to 3000 bytes on bare-metal ports. Performance is
marginally better on a few tests (eg about 1% better on misc_pystone.py and
misc_raytrace.py on PYBv1.1), but overall marginally worse by a percent or
so.
See issue #7542 for further analysis and discussion.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This will always have the maximum/minimum size of a mp_obj_type_t
representation and can be used as a member in other structs.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This will allow the structure of mp_obj_type_t to change while keeping the
definition code the same.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
The buffer protocol type only has a single member, and this existing layout
creates problems for the upcoming split/slot-index mp_obj_type_t layout
optimisations.
If we need to make the buffer protocol more sophisticated in the future
either we can rely on the mp_obj_type_t optimisations to just add
additional slots to mp_obj_type_t or re-visit the buffer protocol then.
This change is a no-op in terms of generated code.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
All uses of this are either tiny strings or not-known-to-be-safe.
Update comments for mp_obj_new_str_copy and mp_obj_new_str_of_type.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
The new `mp_obj_new_str_from_utf8_vstr` can be used when you know you
already have a unicode-safe string.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Previously the desired output type was specified. Now make the type part
of the function name. Because this function is used in a few places this
saves code size due to smaller call-site.
This makes `mp_obj_new_str_type_from_vstr` a private function of objstr.c
(which is almost the only place where the output type isn't a compile-time
constant).
This saves ~140 bytes on PYBV11.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This commit adds the bytes methods to bytearray, matching CPython. The
existing implementations of these methods for str/bytes are reused for
bytearray with minor updates to match CPython return types.
For details on the CPython behaviour see
https://docs.python.org/3/library/stdtypes.html#bytes-and-bytearray-operations
The work to merge locals tables for str/bytes/bytearray/array was done by
@jimmo. Because of this merging of locals the change in code size for this
commit is mostly negative:
bare-arm: +0 +0.000%
minimal x86: +29 +0.018%
unix x64: -792 -0.128% standard[incl -448(data)]
unix nanbox: -436 -0.078% nanbox[incl -448(data)]
stm32: -40 -0.010% PYBV10
cc3200: -32 -0.017%
esp8266: -28 -0.004% GENERIC
esp32: -72 -0.005% GENERIC[incl -200(data)]
mimxrt: -40 -0.011% TEENSY40
renesas-ra: -40 -0.006% RA6M2_EK
nrf: -16 -0.009% pca10040
rp2: -64 -0.013% PICO
samd: +148 +0.105% ADAFRUIT_ITSYBITSY_M4_EXPRESS
This adds new compile-time infrastructure to parse source code files for
`MP_REGISTER_ROOT_POINTER()` and generates a new `root_pointers.h` header
file containing the collected declarations. This works the same as the
existing `MP_REGISTER_MODULE()` feature.
Signed-off-by: David Lechner <david@pybricks.com>
Zero effect on non debug builds, and also usually optimized out even in
debug builds as mp_obj_is_type() is called with a compile-time known type.
I'm not sure we even have dynamic uses of mp_obj_is_type() at the moment,
but if we ever will they will be protected from now on.
Signed-off-by: Yonatan Goldschmidt <yon.goldschmidt@gmail.com>
Commit d96cfd13e3 introduced a regression by breaking existing
users of mp_obj_is_type(.., &mp_obj_bool). This function (and associated
helpers like mp_obj_is_int()) have some specific nuances, and mistakes like
this one can happen again.
This commit adds mp_obj_is_exact_type() which behaves like the the old
mp_obj_is_type(). The new mp_obj_is_type() has the same prototype but it
attempts to statically assert that it's not called with types which should
be checked using mp_obj_is_type(). If called with any of these types: int,
str, bool, NoneType - it will cause a compilation error. Additional
checked types (e.g function types) can be added in the future.
Existing users of mp_obj_is_type() with the now "invalid" types, were
translated to use mp_obj_is_exact_type().
The use of MP_STATIC_ASSERT() is not bulletproof - usually GCC (and other
compilers) can't statically check conditions that are only known during
link-time (like variables' addresses comparison). However, in this case,
GCC is able to statically detect these conditions, probably because it's
the exact same object - `&mp_type_int == &mp_type_int` is detected.
Misuses of this function with runtime-chosen types (e.g:
`mp_obj_type_t *x = ...; mp_obj_is_type(..., x);` won't be detected. MSC
is unable to detect this, so we use MP_STATIC_ASSERT_NOT_MSC().
Compiling with this commit and without the fix for d96cfd13e3 shows
that it detects the problem.
Signed-off-by: Yonatan Goldschmidt <yon.goldschmidt@gmail.com>
It's no longer needed because this macro is now processed after
preprocessing the source code via cpp (in the qstr extraction stage), which
means unused MP_REGISTER_MODULE's are filtered out by the preprocessor.
Signed-off-by: Damien George <damien@micropython.org>
