super().__new__() different inheriting from object vs custom class
super().new() is acting differently when (implicitly) inheriting from object versus (explicitly) inheriting from a custom class. This is different from cPython. Initially discovered and reported for circuitpython (I do not have current access to micropython), but the response there said the same symptoms were seen with micropython (MicroPython v1.22.1 on 2024-01-05; PYBv1.1 with STM32F405RG) using my test code. See https://github.com/adafruit/circuitpython/issues/8939
That report focused on some differences in attribute shadowing. Here I switch to showing the differences in raised TypeErrors. I read the python differences information, and did searches of the issue history ("new"). I found a few very old the looked like they might be linked, but they were marked as bug, complete or notimpel. Mostly all of those. Is this a bug, or one of the expected differences that I could not find documentation for?
# bug_test_subclass_super.py
"""check syntax of super().__new__() in different contexts and implementations"""
# pylint:disable=too-few-public-methods
import sys
import traceback
class C1a:
"""base: works everwhere"""
instance = None
def __new__(cls, *args, **kwargs):
if cls.instance is None:
cls.instance = super().__new__(cls, *args, **kwargs)
return cls.instance
class C1b:
"""base: fails everywhere"""
instance = None
def __new__(cls, *args, **kwargs):
if cls.instance is None:
cls.instance = super().__new__(*args, **kwargs)
return cls.instance
class C2a(C1a):
"""sub: works for cpython"""
def __new__(cls, *args, **kwargs):
if C1a.instance is None:
super().__new__(C1a, *args, **kwargs)
return C1a.instance
class C2b(C1a):
"""sub: works for circuitpython/micropython"""
def __new__(cls, *args, **kwargs):
if C1a.instance is None:
super().__new__(*args, **kwargs)
return C1a.instance
def rpt_exc(context, exc):
"""consistent reporting"""
print(f'{context} instantiation failed:')
# Be compatible with cPython 3.5 and newer
tb_str = traceback.format_exception(None, exc, exc.__traceback__)
print(''.join(tb_str))
def bug_main():
"""run the test"""
print(f'bug_main: start for {sys.implementation}')
print('\nC1a')
tst_in = C1a()
C1a.instance = None
try:
print('C1b')
tst_in = C1b()
except TypeError as exc:
rpt_exc('C1b', exc)
C1a.instance = None
try:
print('C2a')
tst_in = C2a()
except TypeError as exc:
rpt_exc('C2a', exc)
C1a.instance = None
try:
print('C2b')
tst_in = C2b()
except TypeError as exc:
rpt_exc('C2b', exc)
C1a.instance = None
if __name__ == '__main__':
bug_main()
Output for circuitpython
>>> from bug_test_subclass_super import bug_main; bug_main()
bug_main: start for (name='circuitpython', version=(8, 2, 9), mpy=517)
C1a
C1b
C1b instantiation failed:
Traceback (most recent call last):
File "bug_test_subclass_super.py", line 57, in bug_main
File "bug_test_subclass_super.py", line 22, in __new__
TypeError: function takes 1 positional arguments but 0 were given
C2a
C2a instantiation failed:
Traceback (most recent call last):
File "bug_test_subclass_super.py", line 63, in bug_main
File "bug_test_subclass_super.py", line 29, in __new__
File "bug_test_subclass_super.py", line 13, in __new__
TypeError: function takes 1 positional arguments but 2 were given
C2b
output for python 3.7
% python _hpd/bug_test_subclass_super.py
bug_main: start for namespace(_multiarch='x86_64-linux-gnu', cache_tag='cpython-37', hexversion=50794992, name='cpython', version=sys.version_info(major=3, minor=7, micro=17, releaselevel='final', serial=0))
C1a
C1b
C1b instantiation failed:
Traceback (most recent call last):
File "_hpd/bug_test_subclass_super.py", line 57, in bug_main
tst_in = C1b()
File "_hpd/bug_test_subclass_super.py", line 22, in __new__
cls.instance = super().__new__(*args, **kwargs)
TypeError: object.__new__(): not enough arguments
C2a
C2b
C2b instantiation failed:
Traceback (most recent call last):
File "_hpd/bug_test_subclass_super.py", line 69, in bug_main
tst_in = C2b()
File "_hpd/bug_test_subclass_super.py", line 36, in __new__
super().__new__(*args, **kwargs)
TypeError: __new__() missing 1 required positional argument: 'cls'
output for python 3.11
% python _hpd/bug_test_subclass_super.py
bug_main: start for namespace(name='cpython', cache_tag='cpython-311', version=sys.version_info(major=3, minor=11, micro=7, releaselevel='final', serial=0), hexversion=51054576, _multiarch='x86_64-linux-gnu')
C1a
C1b
C1b instantiation failed:
Traceback (most recent call last):
File "/home/phil/development/workspace/circuitpython/space_clock/_hpd/bug_test_subclass_super.py", line 57, in bug_main
tst_in = C1b()
^^^^^
File "/home/phil/development/workspace/circuitpython/space_clock/_hpd/bug_test_subclass_super.py", line 22, in __new__
cls.instance = super().__new__(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: object.__new__(): not enough arguments
C2a
C2b
C2b instantiation failed:
Traceback (most recent call last):
File "/home/phil/development/workspace/circuitpython/space_clock/_hpd/bug_test_subclass_super.py", line 69, in bug_main
tst_in = C2b()
^^^^^
File "/home/phil/development/workspace/circuitpython/space_clock/_hpd/bug_test_subclass_super.py", line 36, in __new__
super().__new__(*args, **kwargs)
TypeError: C1a.__new__() missing 1 required positional argument: 'cls'
tests/cpydiff/core_class: Document issue with super in classmethod.
Summary
This documents an existing problem with the way super() is implemented that became evident while I was implementing __init_subclass__, where when used inside a classmethod it defers to the ancestor of the class object itself (i.e. type), rather than parent classes it inherits from.
This is probably an issue with super itself not treating the second argument differently when its a type in the way that CPython does.
Testing
I've verified this against CPython versions 3.8, 3.10, and 3.12.