Discussion of Python 3.7 support
This issue is intended to track the status of Python 3.7 core features as implemented by MicroPython. Not all of these changes should necessarily be implemented in MicroPython but documenting their status is important.
Python 3.7.0 (final) was released on the 27 June 2018. The Features for 3.7 are defined in PEP 537 and an explanation of the changes can be found in What's New in Python 3.7.
- PEP 538 - Coercing the legacy C locale to a UTF-8 based locale
- PEP 539 - A New C-API for Thread-Local Storage in CPython
- PEP 540 -
UTF-8mode - PEP 552 - Deterministic
pyc - PEP 553 - Built-in breakpoint()
- PEP 557 - Data Classes
- PEP 560 - Core support for typing module and generic types
- PEP 562 - Module
__getattr__and__dir__; see partial implementation (__getattr__): 454cca6016afc96deb6d1ad5d1b3553ab9ad18dd - PEP 563 - Postponed Evaluation of Annotations
- PEP 564 - Time functions with nanosecond resolution; see partial implementation: d4b61b00172ccc231307e3ef33f66f28cb6b051f
- PEP 565 - Show DeprecationWarning in
__main__ - PEP 567 - Context Variables
Other language changes
-
asyncandawaitare now reserved keywords -
dictobjects must preserve insertion-order - More than 255 arguments can now be passed to a function, and a function can now have more than 255 parameters
-
bytes.fromhex()andbytearray.fromhex()now ignore all ASCII whitespace, not only spaces -
str,bytes, andbytearraygained support for the newisascii()method, which can be used to test if a string or bytes contain only the ASCII characters -
ImportErrornow displays module name and module__file__path whenfrom ... import ... fails - Circular imports involving absolute imports with binding a submodule to a name are now supported
-
object.__format__(x, '')is now equivalent tostr(x)rather thanformat(str(self), '') - In order to better support dynamic creation of stack traces,
types.TracebackTypecan now be instantiated from Python code, and thetb_nextattribute on tracebacks is now writable - When using the
-mswitch,sys.path[0]is now eagerly expanded to the full starting directory path, rather than being left as the empty directory (which allows imports from the current working directory at the time when an import occurs) - The new
-X importtimeoption or thePYTHONPROFILEIMPORTTIMEenvironment variable can be used to show the timing of each module import
Changes to MicroPython built-in modules
- asyncio (many, may need a separate ticket)
- gc - New features:
gc.freeze(),gc.unfreeze(),gc-get_freeze_count - math -
math.remainder()added to implement IEEE 754-style remainder - re - A number of tidy up features including better support for splitting on empty strings and copy support for compiled expressions and match objects
- sys -
sys.breakpointhook()added.sys.get(/set)_coroutine_origin_tracking_depth()added. - time - Mostly updates to support nanosecond resolution in PEP564, see above.
(Changes to non-built-in modules will need to be documented elsewhere.)
[WIP] PEP 3115 `metaclass` and PEP 487 `__init_subclass__`
This PR aims to adds optional support for two Python features to MicroPython.
- PEP 3115: Metaclass support with
metaclass=keyword syntax - PEP 487:
__init_subclass__automatic hook with full **kwargs support
The goal of this PR is to discuss if either or both of the PEPs are worth the trade-off of features versus firmware size.
Note : Based on the code size report there are some unguarded features that impact code size that will need to be addressed
These would enable:
- Modern Python class features for MicroPython with minimal overhead
- Three configuration options for different resource constraints
- Full CPython compatibility when enabled (PEP 3115 + PEP 487)
- Zero impact when disabled - all features behind config guards
- Comprehensive test coverage with graceful feature detection
- Enables typing module - Supports
@dataclass_transformdecorator - Enables dataclasses - Foundation for future dataclass implementation
- Enables generic types - User-defined generics via metaclasses
- Industry standard patterns - Plugins, registries now
- Minimal cost - Only 0.6% size increase for full modern class features
All features are intended to be optional, guarded by configuration flags, and designed for minimal code size impact while maintaining near full CPython compatibility.
Modified Files
-
py/mpconfig.h
- Added
MICROPY_METACLASS,MICROPY_METACLASS_PREPARE,MICROPY_INIT_SUBCLASSflags
- Added
-
py/modbuiltins.c
- Modified
__build_class__to accept keyword arguments - Implemented metaclass resolution algorithm (PEP 3115)
- Added optional
__prepare__method support - Extract and filter kwargs for metaclass and
__init_subclass__
- Modified
-
py/objtype.c
- Modified
mp_obj_new_typeto accept metaclass and kw_args parameters - Added
__init_subclass__invocation logic (PEP 487) - Added base class validation for custom metaclasses
- Modified
-
py/vm.c
- Updated LOAD_ATTR fast path to handle type objects with custom metaclasses
-
py/runtime.c
- Fixed error message generation for custom metaclasses
Code Size Impact (Unix standard variant)
Comprehensive measurements across all configuration permutations:
| Configuration | METACLASS | PREPARE | INIT_SUBCLASS | text (bytes) | Total (bytes) | Increase | % Increase |
|---|---|---|---|---|---|---|---|
| Baseline | 0 | 0 | 0 | 177,717 | 193,741 | - | - |
| METACLASS only | 1 | 0 | 0 | 178,341 | 194,365 | +624 | +0.32% |
| METACLASS + PREPARE | 1 | 1 | 0 | 178,493 | 194,517 | +776 | +0.40% |
| INIT_SUBCLASS only | 0 | 0 | 1 | 178,477 | 194,501 | +760 | +0.39% |
| METACLASS + INIT_SUBCLASS ⭐ | 1 | 0 | 1 | 178,909 | 194,933 | +1,192 | +0.62% |
| All features | 1 | 1 | 1 | 179,093 | 195,117 | +1,376 | +0.71% |
⭐ Recommended configuration for dataclass_transform support
Features by Configuration
<details><summary>Details</summary>
<p>
Configuration 1: METACLASS Only (+672 bytes)
#define MICROPY_METACLASS (1)
#define MICROPY_METACLASS_PREPARE (0)
#define MICROPY_INIT_SUBCLASS (0)
Features:
- ✅
class C(metaclass=Meta):syntax - ✅ Metaclass inheritance from base classes
- ✅ Metaclass conflict detection and resolution
- ✅ Metaclass
__init__customization - ✅ Multi-level metaclass inheritance
- ❌ No
__prepare__method - ❌ No
__init_subclass__support
Use Case: Applications that need custom metaclasses but not dataclass patterns.
Configuration 2: METACLASS + PREPARE (800 bytes)
#define MICROPY_METACLASS (1)
#define MICROPY_METACLASS_PREPARE (1)
#define MICROPY_INIT_SUBCLASS (0)
Additional Features:
- ✅
__prepare__method support for custom namespace initialization
Use Case: Full PEP 3115 compliance (rare requirement in embedded systems).
Configuration 3: INIT_SUBCLASS Only (728 bytes)
#define MICROPY_METACLASS (0)
#define MICROPY_METACLASS_PREPARE (0)
#define MICROPY_INIT_SUBCLASS (1)
Features:
- ✅ Automatic
__init_subclass__invocation - ✅ Full **kwargs support (enables dataclass_transform)
- ✅ Implicit classmethod behavior
- ✅ First-base-only invocation per PEP 487
- ✅ Multiple inheritance with
super()chaining - ❌ No
metaclass=keyword syntax
Use Case: Applications needing dataclass_transform but not explicit metaclasses. BEST SIZE/FUNCTIONALITY RATIO for dataclasses.
Configuration 4: METACLASS + INIT_SUBCLASS (+1,152 bytes) ⭐
#define MICROPY_METACLASS (1)
#define MICROPY_METACLASS_PREPARE (0)
#define MICROPY_INIT_SUBCLASS (1)
Features:
- ✅ All features from Config 1 and Config 3 combined
- ✅ Full dataclass_transform support
- ✅ Complete typing module patterns
- ❌ No
__prepare__(rarely needed)
Use Case: RECOMMENDED - Full modern Python class features for dataclasses and type systems.
Configuration 5: All Features (+1,248 bytes)
#define MICROPY_METACLASS (1)
#define MICROPY_METACLASS_PREPARE (1)
#define MICROPY_INIT_SUBCLASS (1)
Features:
- ✅ Complete PEP 3115 + PEP 487 implementation
- ✅ Full CPython compatibility
Use Case: Maximum compatibility, when code size is less critical.
</p>
</details>
Trade-Off Analysis
Size contributions
Sizes measured on unix standard build using mpbuild :
| Config | Size | features | diff | diff | diff | total |
|---|---|---|---|---|---|---|
| standard | 683564 | 0 0 0 | - | |||
| 1 | 684236 | 1 0 0 | 672 | 672 | ||
| 2 | 684364 | 1 1 0 | 672 | 128 | 800 | |
| 3 | 684292 | 0 0 1 | 728 | 728 | ||
| 4 | 684716 | 1 0 1 | 424 | 728 | 1152 | |
| 5 | 684812 | 1 1 1 | 672 | 128 | 448 | 1248 |
Size Savings Options
| Option | Configuration | Bytes Saved | Trade-Off |
|---|---|---|---|
| Skip prepare | (1,0,1) vs (1,1,1) | -128 | Loses rarely-used feature, no practical impact |
| Skip METACLASS | (0,0,1) vs (1,0,1) | -424 | Loses metaclass= syntax, keeps dataclass support |
| Skip INIT_SUBCLASS | (1,0,0) vs (1,0,1) | -480 | Loses dataclass_transform, keeps metaclasses |
Alternatives:
Several additional optimization strategies were explored:
-
Remove kwargs support from init_subclass
- Saves: ~200-250 bytes
- BREAKS: dataclass_transform pattern (primary use case)
- Verdict: Not viable
-
Simplify metaclass resolution
- Saves: ~80-100 bytes
- BREAKS: Proper conflict detection, multiple inheritance
- Verdict: Compromises correctness
-
Stack-based allocation
- INCREASES size by +160 bytes due to conditional logic overhead
- Verdict: Counter-productive
-
Additional granular flags
- Adding more config flags for sub-features increases overhead
- Shared code paths mean splitting would duplicate code
- Verdict: Makes codebase more complex without real savings
Code Sharing Benefits
The current implementation benefits from significant code sharing:
- Keyword extraction logic shared between metaclass and init_subclass
- Metaclass resolution reuses type checking infrastructure
- Memory management uses existing m_new/m_del inline functions
- Combined features cost less than sum of parts
Configuration Recommendations
<details><summary>Details</summary>
<p>
Embedded/Resource-Constrained (+728 bytes)
Configuration 3: INIT_SUBCLASS Only
#define MICROPY_METACLASS (0)
#define MICROPY_INIT_SUBCLASS (1)
Rationale:
-
Enables modern dataclass patterns with minimal overhead
-
Best size-to-functionality ratio
-
Most embedded applications don't need explicit metaclass syntax
-
User Defined Generics not possible without METACLASS.
General Purpose (+1,152 bytes)
Configuration 4: METACLASS + INIT_SUBCLASS ⭐
#define MICROPY_METACLASS (1)
#define MICROPY_INIT_SUBCLASS (1)
Rationale:
- Complete modern Python class features
- Supports typing module, dataclasses, and custom metaclasses
- Only 0.6% size increase
- Recommended default for ports with >256KB flash
Maximum Compatibility (+1,248 bytes)
Configuration 5: All Features
#define MICROPY_METACLASS (1)
#define MICROPY_METACLASS_PREPARE (1)
#define MICROPY_INIT_SUBCLASS (1)
Rationale:
- Full PEP 3115 + PEP 487 compliance
- For desktop/high-resource targets
- Ensures maximum CPython code compatibility
Minimal (Baseline, +0 bytes)
Configuration: All Disabled (Default)
#define MICROPY_METACLASS (0)
#define MICROPY_INIT_SUBCLASS (0)
Rationale:
- Ultra-constrained targets (<128KB flash)
- Simple applications without advanced class features
- Default to maintain backward compatibility
</p>
</details>
Usage Examples
<details><summary>Details</summary>
<p>
Example 1: Metaclass (Requires MICROPY_METACLASS=1)
class Meta(type):
def __init__(cls, name, bases, dct):
cls.auto_added = 'by metaclass'
cls.attr_count = len(dct)
class Base(metaclass=Meta):
x = 1
y = 2
class Derived(Base): # Inherits Meta as metaclass
z = 3
print(Base.auto_added) # 'by metaclass'
print(Base.attr_count) # 2
print(Derived.auto_added) # 'by metaclass'
print(Derived.attr_count) # 1
print(type(Derived).__name__) # 'Meta'
Example 2: dataclass_transform Pattern (Requires MICROPY_INIT_SUBCLASS=1)
class ModelBase:
def __init_subclass__(cls, *, init=True, frozen=False, eq=True, order=True):
# Note: No @classmethod decorator needed (implicit per PEP 487)
cls._config = {
'init': init,
'frozen': frozen,
'eq': eq,
'order': order
}
# Generate methods based on config...
if init:
cls._generate_init()
if eq:
cls._generate_eq()
class CustomerModel(
ModelBase,
init=False,
frozen=True,
eq=False,
order=False,
):
id: int
name: str
print(CustomerModel._config)
# {'init': False, 'frozen': True, 'eq': False, 'order': False}
Example 3: Plugin Registry (Requires MICROPY_INIT_SUBCLASS=1)
class PluginBase:
_plugins = []
def __init_subclass__(cls, **kwargs):
super().__init_subclass__(**kwargs)
PluginBase._plugins.append(cls)
class MyPlugin(PluginBase):
pass
class AnotherPlugin(PluginBase):
pass
print(len(PluginBase._plugins)) # 2
print(MyPlugin in PluginBase._plugins) # True
Example 4: Multiple Inheritance with Chaining (Requires MICROPY_INIT_SUBCLASS=1)
class BaseA:
def __init_subclass__(cls, **kwargs):
cls.from_a = True
super().__init_subclass__(**kwargs) # Chain to next base
class BaseB:
def __init_subclass__(cls, **kwargs):
cls.from_b = True
super().__init_subclass__(**kwargs)
class Multi(BaseA, BaseB):
pass
# Per PEP 487: Only first base's __init_subclass__ is called
# BaseA chains to BaseB via super()
print(hasattr(Multi, 'from_a')) # True
print(hasattr(Multi, 'from_b')) # True (via chaining)
</p>
</details>
Testing
- tests/basics/class_metaclass.py - 6 comprehensive metaclass tests
- tests/basics/class_init_subclass.py - 7 init_subclass tests with kwargs
- tests/cpydiff/core_class_metaclass.py - Documents
__prepare__limitation
All tests gracefully skip when features are disabled.
Compatibility
- Backward compatible - No breaking changes, features disabled by default
- Tested across configurations - All feature combinations verified
- Port-specific configuration - Each port can choose appropriate level
- Standards compliant - Follows PEP 3115 and PEP 487 specifications
Recommended Adoption Strategy
- Phase 1: Enable for Unix and larger ports (METACLASS + INIT_SUBCLASS)
- Phase 2: Enable INIT_SUBCLASS only for medium-resource ports
- Phase 3: Keep disabled for minimal/constrained ports
- Documentation: Update port-specific docs with configuration guidance
Technical Notes
PEP 487 Compliance Notes
__init_subclass__is implicitly a classmethod (no decorator needed)- Only first base's
__init_subclass__is called (per MRO) - Chaining requires explicit
super().__init_subclass__(**kwargs)call - Receives all class kwargs except
metaclass=
PEP 3115 Compliance Notes
metaclass=keyword syntax fully supported- Proper metaclass resolution with conflict detection
- Multi-level metaclass inheritance works correctly
__prepare__method optional (controlled by separate flag)
Known Limitations (by design)
-
When MICROPY_METACLASS_PREPARE=0:
__prepare__method not called- Documented in tests/cpydiff/core_class_metaclass.py
- Rarely needed in embedded applications
-
Decorator behavior:
- Explicit
@classmethodon__init_subclass__not supported - Documented in tests/cpydiff/core_class_initsubclass_classmethod.py
- Not needed per PEP 487 (implicit classmethod)
- Explicit
Performance Impact
- Runtime: Negligible - only affects class creation (not instance operations)
- Memory: No additional RAM usage beyond class structures themselves
Performance Impact Analysis
Runtime performance impact of PEP 3115 (METACLASS) and PEP 487 (INIT_SUBCLASS) features.
Test Platform
- Port: Unix (standard variant)
- Platform: Linux x86_64
- Compiler: GCC with -O2 optimization
- Test Method: Average of 3 runs per configuration
Performance Results
<details><summary>Details</summary>
<p>
| Test | Baseline | METACLASS Only | INIT_SUBCLASS Only | Both Enabled |
|---|---|---|---|---|
| class_creation | 693 µs | 770 µs (+11.1%) | 758 µs (+9.4%) | 707 µs (+2.0%) |
| inheritance | 602 µs | 639 µs (+6.1%) | 642 µs (+6.6%) | 638 µs (+6.0%) |
| method_calls | 620 µs | 581 µs (-6.3%) | 578 µs (-6.8%) | 584 µs (-5.8%) |
| attr_access | 351 µs | 328 µs (-6.6%) | 329 µs (-6.3%) | 323 µs (-8.0%) |
All measurements in microseconds (µs). Percentages show runtime change vs baseline.
Analysis
Class Creation Overhead
- METACLASS only: +11.1% overhead when creating classes
- INIT_SUBCLASS only: +9.4% overhead when creating classes
- Both enabled: +2.0% overhead (shared code paths optimize combined case)
The overhead is expected as the implementation must check for metaclass resolution and invoke __init_subclass__ hooks during class creation.
Inheritance Performance
- Consistent ~6% overhead across all configurations
- Expected due to additional metaclass resolution checks when creating derived classes
- Overhead is consistent and predictable
Method Calls & Attribute Access
- Improvement of 5-8% in both method calls and attribute access
- Counterintuitive result likely due to:
- Better code layout/alignment after adding guarded code
- Compiler optimization differences
- Cache effects from slightly different memory layout
Important: These improvements are within measurement noise and should not be considered a performance benefit. The key finding is that method calls and attribute access show no significant performance degradation.
Summary
Performance Impact
Negligible runtime overhead for typical operations
- Method calls: No degradation (measurement noise)
- Attribute access: No degradation (measurement noise)
- Class creation: 2-11% slower (only during class definition time)
- Inheritance: ~6% slower (only during class definition time)
Key Findings
- Hot path unchanged: Method invocation and attribute access (the most common operations) show no performance regression
- Cold path overhead acceptable: Class creation overhead only matters during module import/initialization
- Combined features optimized: Enabling both features together has less overhead than individual features due to shared code paths
Recommendations
- For production code: Performance impact is minimal since class creation happens at module import time
- For dynamic class generation: 2-11% overhead on class creation is acceptable for the additional functionality gained
- Overall assessment: The runtime performance impact is negligible for typical MicroPython applications
Performance Test Descriptions
- class_creation: Creating 500 simple classes with methods (measures class construction overhead)
- inheritance: Creating 500 derived classes (measures inheritance + metaclass resolution overhead)
- method_calls: 5000 method invocations (measures hot path performance)
- attr_access: 5000 attribute accesses (measures attribute lookup performance)
</p>
</details>