← index #7886PR #18362
Related · medium · value 0.523
QUERY · ISSUE

Discussion of Python 3.7 support

openby mattytrentiniopened 2021-10-08updated 2025-10-03
py-core

This issue is intended to track the status of Python 3.7 core features as implemented by MicroPython. Not all of these changes should necessarily be implemented in MicroPython but documenting their status is important.

Python 3.7.0 (final) was released on the 27 June 2018. The Features for 3.7 are defined in PEP 537 and an explanation of the changes can be found in What's New in Python 3.7.

  • PEP 538 - Coercing the legacy C locale to a UTF-8 based locale
  • PEP 539 - A New C-API for Thread-Local Storage in CPython
  • PEP 540 - UTF-8 mode
  • PEP 552 - Deterministic pyc
  • PEP 553 - Built-in breakpoint()
  • PEP 557 - Data Classes
  • PEP 560 - Core support for typing module and generic types
  • PEP 562 - Module __getattr__ and __dir__; see partial implementation (__getattr__): 454cca6016afc96deb6d1ad5d1b3553ab9ad18dd
  • PEP 563 - Postponed Evaluation of Annotations
  • PEP 564 - Time functions with nanosecond resolution; see partial implementation: d4b61b00172ccc231307e3ef33f66f28cb6b051f
  • PEP 565 - Show DeprecationWarning in __main__
  • PEP 567 - Context Variables

Other language changes

  • async and await are now reserved keywords
  • dict objects must preserve insertion-order
  • More than 255 arguments can now be passed to a function, and a function can now have more than 255 parameters
  • bytes.fromhex() and bytearray.fromhex() now ignore all ASCII whitespace, not only spaces
  • str, bytes, and bytearray gained support for the new isascii() method, which can be used to test if a string or bytes contain only the ASCII characters
  • ImportError now displays module name and module __file__ path when from ... import ... fails
  • Circular imports involving absolute imports with binding a submodule to a name are now supported
  • object.__format__(x, '') is now equivalent to str(x) rather than format(str(self), '')
  • In order to better support dynamic creation of stack traces, types.TracebackType can now be instantiated from Python code, and the tb_next attribute on tracebacks is now writable
  • When using the -m switch, sys.path[0] is now eagerly expanded to the full starting directory path, rather than being left as the empty directory (which allows imports from the current working directory at the time when an import occurs)
  • The new -X importtime option or the PYTHONPROFILEIMPORTTIME environment variable can be used to show the timing of each module import

Changes to MicroPython built-in modules

  • asyncio (many, may need a separate ticket)
  • gc - New features: gc.freeze(), gc.unfreeze(), gc-get_freeze_count
  • math - math.remainder() added to implement IEEE 754-style remainder
  • re - A number of tidy up features including better support for splitting on empty strings and copy support for compiled expressions and match objects
  • sys - sys.breakpointhook() added. sys.get(/set)_coroutine_origin_tracking_depth() added.
  • time - Mostly updates to support nanosecond resolution in PEP564, see above.

(Changes to non-built-in modules will need to be documented elsewhere.)

CANDIDATE · PULL REQUEST

[WIP] PEP 3115 `metaclass` and PEP 487 `__init_subclass__`

closedby Josverlopened 2025-11-03updated 2026-02-20
enhancementpy-core

This PR aims to adds optional support for two Python features to MicroPython.

  • PEP 3115: Metaclass support with metaclass= keyword syntax
  • PEP 487: __init_subclass__ automatic hook with full **kwargs support

The goal of this PR is to discuss if either or both of the PEPs are worth the trade-off of features versus firmware size.

Note : Based on the code size report there are some unguarded features that impact code size that will need to be addressed

These would enable:

  1. Modern Python class features for MicroPython with minimal overhead
  2. Three configuration options for different resource constraints
  3. Full CPython compatibility when enabled (PEP 3115 + PEP 487)
  4. Zero impact when disabled - all features behind config guards
  5. Comprehensive test coverage with graceful feature detection
  • Enables typing module - Supports @dataclass_transform decorator
  • Enables dataclasses - Foundation for future dataclass implementation
  • Enables generic types - User-defined generics via metaclasses
  • Industry standard patterns - Plugins, registries now
  • Minimal cost - Only 0.6% size increase for full modern class features

All features are intended to be optional, guarded by configuration flags, and designed for minimal code size impact while maintaining near full CPython compatibility.

Modified Files

  1. py/mpconfig.h

    • Added MICROPY_METACLASS, MICROPY_METACLASS_PREPARE, MICROPY_INIT_SUBCLASS flags
  2. py/modbuiltins.c

    • Modified __build_class__ to accept keyword arguments
    • Implemented metaclass resolution algorithm (PEP 3115)
    • Added optional __prepare__ method support
    • Extract and filter kwargs for metaclass and __init_subclass__
  3. py/objtype.c

    • Modified mp_obj_new_type to accept metaclass and kw_args parameters
    • Added __init_subclass__ invocation logic (PEP 487)
    • Added base class validation for custom metaclasses
  4. py/vm.c

    • Updated LOAD_ATTR fast path to handle type objects with custom metaclasses
  5. py/runtime.c

    • Fixed error message generation for custom metaclasses

Code Size Impact (Unix standard variant)

Comprehensive measurements across all configuration permutations:

Configuration METACLASS PREPARE INIT_SUBCLASS text (bytes) Total (bytes) Increase % Increase
Baseline 0 0 0 177,717 193,741 - -
METACLASS only 1 0 0 178,341 194,365 +624 +0.32%
METACLASS + PREPARE 1 1 0 178,493 194,517 +776 +0.40%
INIT_SUBCLASS only 0 0 1 178,477 194,501 +760 +0.39%
METACLASS + INIT_SUBCLASS 1 0 1 178,909 194,933 +1,192 +0.62%
All features 1 1 1 179,093 195,117 +1,376 +0.71%

Recommended configuration for dataclass_transform support

Features by Configuration

<details><summary>Details</summary>
<p>

Configuration 1: METACLASS Only (+672 bytes)

#define MICROPY_METACLASS (1)
#define MICROPY_METACLASS_PREPARE (0)
#define MICROPY_INIT_SUBCLASS (0)

Features:

  • class C(metaclass=Meta): syntax
  • ✅ Metaclass inheritance from base classes
  • ✅ Metaclass conflict detection and resolution
  • ✅ Metaclass __init__ customization
  • ✅ Multi-level metaclass inheritance
  • ❌ No __prepare__ method
  • ❌ No __init_subclass__ support

Use Case: Applications that need custom metaclasses but not dataclass patterns.

Configuration 2: METACLASS + PREPARE (800 bytes)

#define MICROPY_METACLASS (1)
#define MICROPY_METACLASS_PREPARE (1)
#define MICROPY_INIT_SUBCLASS (0)

Additional Features:

  • __prepare__ method support for custom namespace initialization

Use Case: Full PEP 3115 compliance (rare requirement in embedded systems).

Configuration 3: INIT_SUBCLASS Only (728 bytes)

#define MICROPY_METACLASS (0)
#define MICROPY_METACLASS_PREPARE (0)
#define MICROPY_INIT_SUBCLASS (1)

Features:

  • ✅ Automatic __init_subclass__ invocation
  • ✅ Full **kwargs support (enables dataclass_transform)
  • ✅ Implicit classmethod behavior
  • ✅ First-base-only invocation per PEP 487
  • ✅ Multiple inheritance with super() chaining
  • ❌ No metaclass= keyword syntax

Use Case: Applications needing dataclass_transform but not explicit metaclasses. BEST SIZE/FUNCTIONALITY RATIO for dataclasses.

Configuration 4: METACLASS + INIT_SUBCLASS (+1,152 bytes) ⭐

#define MICROPY_METACLASS (1)
#define MICROPY_METACLASS_PREPARE (0)
#define MICROPY_INIT_SUBCLASS (1)

Features:

  • ✅ All features from Config 1 and Config 3 combined
  • ✅ Full dataclass_transform support
  • ✅ Complete typing module patterns
  • ❌ No __prepare__ (rarely needed)

Use Case: RECOMMENDED - Full modern Python class features for dataclasses and type systems.

Configuration 5: All Features (+1,248 bytes)

#define MICROPY_METACLASS (1)
#define MICROPY_METACLASS_PREPARE (1)
#define MICROPY_INIT_SUBCLASS (1)

Features:

  • ✅ Complete PEP 3115 + PEP 487 implementation
  • ✅ Full CPython compatibility

Use Case: Maximum compatibility, when code size is less critical.

</p>
</details>

Trade-Off Analysis

Size contributions

Sizes measured on unix standard build using mpbuild :

Config Size features diff diff diff total
standard 683564 0 0 0 -
1 684236 1 0 0 672 672
2 684364 1 1 0 672 128 800
3 684292 0 0 1 728 728
4 684716 1 0 1 424 728 1152
5 684812 1 1 1 672 128 448 1248

Size Savings Options

Option Configuration Bytes Saved Trade-Off
Skip prepare (1,0,1) vs (1,1,1) -128 Loses rarely-used feature, no practical impact
Skip METACLASS (0,0,1) vs (1,0,1) -424 Loses metaclass= syntax, keeps dataclass support
Skip INIT_SUBCLASS (1,0,0) vs (1,0,1) -480 Loses dataclass_transform, keeps metaclasses

Alternatives:

Several additional optimization strategies were explored:

  1. Remove kwargs support from init_subclass

    • Saves: ~200-250 bytes
    • BREAKS: dataclass_transform pattern (primary use case)
    • Verdict: Not viable
  2. Simplify metaclass resolution

    • Saves: ~80-100 bytes
    • BREAKS: Proper conflict detection, multiple inheritance
    • Verdict: Compromises correctness
  3. Stack-based allocation

    • INCREASES size by +160 bytes due to conditional logic overhead
    • Verdict: Counter-productive
  4. Additional granular flags

    • Adding more config flags for sub-features increases overhead
    • Shared code paths mean splitting would duplicate code
    • Verdict: Makes codebase more complex without real savings

Code Sharing Benefits

The current implementation benefits from significant code sharing:

  • Keyword extraction logic shared between metaclass and init_subclass
  • Metaclass resolution reuses type checking infrastructure
  • Memory management uses existing m_new/m_del inline functions
  • Combined features cost less than sum of parts

Configuration Recommendations

<details><summary>Details</summary>
<p>

Embedded/Resource-Constrained (+728 bytes)

Configuration 3: INIT_SUBCLASS Only

#define MICROPY_METACLASS (0)
#define MICROPY_INIT_SUBCLASS (1)

Rationale:

  • Enables modern dataclass patterns with minimal overhead

  • Best size-to-functionality ratio

  • Most embedded applications don't need explicit metaclass syntax

  • User Defined Generics not possible without METACLASS.

General Purpose (+1,152 bytes)

Configuration 4: METACLASS + INIT_SUBCLASS

#define MICROPY_METACLASS (1)
#define MICROPY_INIT_SUBCLASS (1)

Rationale:

  • Complete modern Python class features
  • Supports typing module, dataclasses, and custom metaclasses
  • Only 0.6% size increase
  • Recommended default for ports with >256KB flash

Maximum Compatibility (+1,248 bytes)

Configuration 5: All Features

#define MICROPY_METACLASS (1)
#define MICROPY_METACLASS_PREPARE (1)
#define MICROPY_INIT_SUBCLASS (1)

Rationale:

  • Full PEP 3115 + PEP 487 compliance
  • For desktop/high-resource targets
  • Ensures maximum CPython code compatibility

Minimal (Baseline, +0 bytes)

Configuration: All Disabled (Default)

#define MICROPY_METACLASS (0)
#define MICROPY_INIT_SUBCLASS (0)

Rationale:

  • Ultra-constrained targets (<128KB flash)
  • Simple applications without advanced class features
  • Default to maintain backward compatibility

</p>
</details>

Usage Examples

<details><summary>Details</summary>
<p>

Example 1: Metaclass (Requires MICROPY_METACLASS=1)

class Meta(type):
    def __init__(cls, name, bases, dct):
        cls.auto_added = 'by metaclass'
        cls.attr_count = len(dct)

class Base(metaclass=Meta):
    x = 1
    y = 2

class Derived(Base):  # Inherits Meta as metaclass
    z = 3

print(Base.auto_added)      # 'by metaclass'
print(Base.attr_count)      # 2
print(Derived.auto_added)   # 'by metaclass'
print(Derived.attr_count)   # 1
print(type(Derived).__name__)  # 'Meta'

Example 2: dataclass_transform Pattern (Requires MICROPY_INIT_SUBCLASS=1)

class ModelBase:
    def __init_subclass__(cls, *, init=True, frozen=False, eq=True, order=True):
        # Note: No @classmethod decorator needed (implicit per PEP 487)
        cls._config = {
            'init': init,
            'frozen': frozen,
            'eq': eq,
            'order': order
        }
        # Generate methods based on config...
        if init:
            cls._generate_init()
        if eq:
            cls._generate_eq()

class CustomerModel(
    ModelBase,
    init=False,
    frozen=True,
    eq=False,
    order=False,
):
    id: int
    name: str

print(CustomerModel._config)
# {'init': False, 'frozen': True, 'eq': False, 'order': False}

Example 3: Plugin Registry (Requires MICROPY_INIT_SUBCLASS=1)

class PluginBase:
    _plugins = []
    
    def __init_subclass__(cls, **kwargs):
        super().__init_subclass__(**kwargs)
        PluginBase._plugins.append(cls)

class MyPlugin(PluginBase):
    pass

class AnotherPlugin(PluginBase):
    pass

print(len(PluginBase._plugins))  # 2
print(MyPlugin in PluginBase._plugins)  # True

Example 4: Multiple Inheritance with Chaining (Requires MICROPY_INIT_SUBCLASS=1)

class BaseA:
    def __init_subclass__(cls, **kwargs):
        cls.from_a = True
        super().__init_subclass__(**kwargs)  # Chain to next base

class BaseB:
    def __init_subclass__(cls, **kwargs):
        cls.from_b = True
        super().__init_subclass__(**kwargs)

class Multi(BaseA, BaseB):
    pass

# Per PEP 487: Only first base's __init_subclass__ is called
# BaseA chains to BaseB via super()
print(hasattr(Multi, 'from_a'))  # True
print(hasattr(Multi, 'from_b'))  # True (via chaining)

</p>
</details>

Testing

  • tests/basics/class_metaclass.py - 6 comprehensive metaclass tests
  • tests/basics/class_init_subclass.py - 7 init_subclass tests with kwargs
  • tests/cpydiff/core_class_metaclass.py - Documents __prepare__ limitation

All tests gracefully skip when features are disabled.

Compatibility

  • Backward compatible - No breaking changes, features disabled by default
  • Tested across configurations - All feature combinations verified
  • Port-specific configuration - Each port can choose appropriate level
  • Standards compliant - Follows PEP 3115 and PEP 487 specifications

Recommended Adoption Strategy

  1. Phase 1: Enable for Unix and larger ports (METACLASS + INIT_SUBCLASS)
  2. Phase 2: Enable INIT_SUBCLASS only for medium-resource ports
  3. Phase 3: Keep disabled for minimal/constrained ports
  4. Documentation: Update port-specific docs with configuration guidance

Technical Notes

PEP 487 Compliance Notes

  • __init_subclass__ is implicitly a classmethod (no decorator needed)
  • Only first base's __init_subclass__ is called (per MRO)
  • Chaining requires explicit super().__init_subclass__(**kwargs) call
  • Receives all class kwargs except metaclass=

PEP 3115 Compliance Notes

  • metaclass= keyword syntax fully supported
  • Proper metaclass resolution with conflict detection
  • Multi-level metaclass inheritance works correctly
  • __prepare__ method optional (controlled by separate flag)

Known Limitations (by design)

  1. When MICROPY_METACLASS_PREPARE=0:

    • __prepare__ method not called
    • Documented in tests/cpydiff/core_class_metaclass.py
    • Rarely needed in embedded applications
  2. Decorator behavior:

    • Explicit @classmethod on __init_subclass__ not supported
    • Documented in tests/cpydiff/core_class_initsubclass_classmethod.py
    • Not needed per PEP 487 (implicit classmethod)

Performance Impact

  • Runtime: Negligible - only affects class creation (not instance operations)
  • Memory: No additional RAM usage beyond class structures themselves

Performance Impact Analysis

Runtime performance impact of PEP 3115 (METACLASS) and PEP 487 (INIT_SUBCLASS) features.

Test Platform

  • Port: Unix (standard variant)
  • Platform: Linux x86_64
  • Compiler: GCC with -O2 optimization
  • Test Method: Average of 3 runs per configuration

Performance Results

<details><summary>Details</summary>
<p>

Test Baseline METACLASS Only INIT_SUBCLASS Only Both Enabled
class_creation 693 µs 770 µs (+11.1%) 758 µs (+9.4%) 707 µs (+2.0%)
inheritance 602 µs 639 µs (+6.1%) 642 µs (+6.6%) 638 µs (+6.0%)
method_calls 620 µs 581 µs (-6.3%) 578 µs (-6.8%) 584 µs (-5.8%)
attr_access 351 µs 328 µs (-6.6%) 329 µs (-6.3%) 323 µs (-8.0%)

All measurements in microseconds (µs). Percentages show runtime change vs baseline.

Analysis

Class Creation Overhead

  • METACLASS only: +11.1% overhead when creating classes
  • INIT_SUBCLASS only: +9.4% overhead when creating classes
  • Both enabled: +2.0% overhead (shared code paths optimize combined case)

The overhead is expected as the implementation must check for metaclass resolution and invoke __init_subclass__ hooks during class creation.

Inheritance Performance

  • Consistent ~6% overhead across all configurations
  • Expected due to additional metaclass resolution checks when creating derived classes
  • Overhead is consistent and predictable

Method Calls & Attribute Access

  • Improvement of 5-8% in both method calls and attribute access
  • Counterintuitive result likely due to:
    • Better code layout/alignment after adding guarded code
    • Compiler optimization differences
    • Cache effects from slightly different memory layout

Important: These improvements are within measurement noise and should not be considered a performance benefit. The key finding is that method calls and attribute access show no significant performance degradation.

Summary

Performance Impact

Negligible runtime overhead for typical operations

  • Method calls: No degradation (measurement noise)
  • Attribute access: No degradation (measurement noise)
  • Class creation: 2-11% slower (only during class definition time)
  • Inheritance: ~6% slower (only during class definition time)

Key Findings

  1. Hot path unchanged: Method invocation and attribute access (the most common operations) show no performance regression
  2. Cold path overhead acceptable: Class creation overhead only matters during module import/initialization
  3. Combined features optimized: Enabling both features together has less overhead than individual features due to shared code paths

Recommendations

  • For production code: Performance impact is minimal since class creation happens at module import time
  • For dynamic class generation: 2-11% overhead on class creation is acceptable for the additional functionality gained
  • Overall assessment: The runtime performance impact is negligible for typical MicroPython applications

Performance Test Descriptions

  • class_creation: Creating 500 simple classes with methods (measures class construction overhead)
  • inheritance: Creating 500 derived classes (measures inheritance + metaclass resolution overhead)
  • method_calls: 5000 method invocations (measures hot path performance)
  • attr_access: 5000 attribute accesses (measures attribute lookup performance)

</p>
</details>

Keyboard

j / / n
next pair
k / / p
previous pair
1 / / h
show query pane
2 / / l
show candidate pane
c
copy suggested comment
r
toggle reasoning
g i
go to index
?
show this help
esc
close overlays

press ? or esc to close

copied