Discussion of Python 3.5 support
MicroPython aims to implement the Python 3.x "standard". At the moment x is currently 4, ie we try to be compatible with CPython 3.4. It's new territory as to what to do when CPython evolves to larger version numbers. I would say we should try to follow the changes and implement them when possible/sensible.
Python 3.5 had a feature freeze on 24 May 2015 and is scheduled for final release on 13 September 2015. This ticket here is about discussing if, what and how we should upgrade uPy to version 3.5 of the language. It may be that some issues need to break off into separate tickets and that's fine but we should link to them from this one.
The PEP discussing the new features in 3.5: https://www.python.org/dev/peps/pep-0478/
A more friendly overview in the docs: https://docs.python.org/3.5/whatsnew/3.5.html
Below is a list of finalised/accepted PEPs for 3.5 grouped into their impact to MicroPython.
Extensions to the syntax:
- PEP 448 - additional unpacking generalizations; see #5807
- PEP 465 - a new matrix multiplication operator; done in 2069c563f9e944d8f45e524b425fff23208e8153
- PEP 492 - coroutines with async and await syntax
Extensions and changes to the runtime:
- PEP 461 - %-formatting for binary strings - tentatively done
- PEP 475 - retrying system calls that fail with EINTR - done in 9418611c8aa860812e28b6e2b6bfde7be5817b43
- PEP 479 - change StopIteration handling inside generators - done in 3f6ffe059f64b3ebc44dc0bbc63452cb8850702b
Standard library changes:
- PEP 471 - os.scandir()
- PEP 485 - math.isclose(), a function for testing approximate equality - done in af5c998f37ddc62abfd36e0b8be511c392fc25d8
Miscellaneous changes that are not relevant to MicroPython:
- PEP 441 - improved Python zip application support
- PEP 486 - make the Python Launcher aware of virtual environments
- PEP 484 - type hints (advisory only)
- PEP 488 - elimination of PYO files
- PEP 489 - redesigning extension module loading
- Added the
"namereplace"error handlers. The"backslashreplace"error handlers now work with decoding and translating. - Property docstrings are now writable. This is especially useful for collections.namedtuple() docstrings.
- Circular imports involving relative imports are now supported.
Changes to MicroPython built-in modules
- asyncio (many, may need another ticket)
- cmath - A new function
isclose()provides a way to test for approximate equality.
- collections
- The
OrderedDictclass is now implemented in C, which makes it 4 to 100 times faster. -
OrderedDict.items(),OrderedDict.keys(),OrderedDict.values()views now supportreversed()iteration. - The deque class now defines
index(),insert(), andcopy(), and supports the + and * operators. - Docstrings produced by
namedtuple()can now be updated. - The
UserStringclass now implements the__getnewargs__(),__rmod__(),casefold(),format_map(),isprintable(), andmaketrans()methods to match the corresponding methods ofstr.
- The
- heapq - Element comparison in
merge()can now be customized by passing a key function in a new optional key keyword argument, and a new optionalreversekeyword argument can be used to reverse element comparison - io - A new
BufferedIOBase.readinto1()method, that uses at most one call to the underlying raw stream'sRawIOBase.read()orRawIOBase.readinto()methods. - json - JSON decoder now raises
JSONDecodeErrorinstead ofValueErrorto provide better context information about the error.
- math
- Two new constants have been added to the math module:
infandnan. - A new function
isclose()provides a way to test for approximate equality. - A new
gcd()function has been added. Thefractions.gcd()function is now deprecated.
- Two new constants have been added to the math module:
- os
- The new
scandir()function returning an iterator ofDirEntryobjects has been added. - The
urandom()function now uses thegetrandom()syscall on Linux 3.17 or newer, and getentropy() on OpenBSD 5.6 and newer, removing the need to use /dev/urandom and avoiding failures due to potential file descriptor exhaustion. - New
get_blocking()andset_blocking()functions allow getting and setting a file descriptor's blocking mode (O_NONBLOCK.) - There is a new
os.path.commonpath()function returning the longest common sub-path of each passed pathname.
- The new
- re
- References and conditional references to groups with fixed length are now allowed in lookbehind assertions.
- The number of capturing groups in regular expressions is no longer limited to 100.
- The
sub()andsubn()functions now replace unmatched groups with empty strings instead of raising an exception. - The
re.errorexceptions have new attributes,msg,pattern,pos,lineno, andcolno, that provide better context information about the error
- socket
- Functions with timeouts now use a monotonic clock, instead of a system clock.
- A new
socket.sendfile()method allows sending a file over a socket by using the high-performanceos.sendfile()function on UNIX, resulting in uploads being from 2 to 3 times faster than when using plainsocket.send(). - The
socket.sendall()method no longer resets the socket timeout every time bytes are received or sent. The socket timeout is now the maximum total duration to send all data. - The backlog argument of the
socket.listen()method is now optional. By default it is set to SOMAXCONN or to 128, whichever is less.
- ssl
- Memory BIO Support
- Application-Layer Protocol Negotiation Support
- There is a new
SSLSocket.version()method to query the actual protocol version in use. - The
SSLSocketclass now implements aSSLSocket.sendfile()method. - The
SSLSocket.send()method now raises either thessl.SSLWantReadErrororssl.SSLWantWriteErrorexception on a non-blocking socket if the operation would block. Previously, it would return 0. - The
cert_time_to_seconds()function now interprets the input time as UTC and not as local time, per RFC 5280. Additionally, the return value is always an int. - New
SSLObject.shared_ciphers()andSSLSocket.shared_ciphers()methods return the list of ciphers sent by the client during the handshake. - The
SSLSocket.do_handshake(),SSLSocket.read(),SSLSocket.shutdown(), andSSLSocket.write()methods of theSSLSocketclass no longer reset the socket timeout every time bytes are received or sent. - The
match_hostname()function now supports matching of IP addresses.
- sys
- A new
set_coroutine_wrapper()function allows setting a global hook that will be called whenever a coroutine object is created by an async def function. A correspondingget_coroutine_wrapper()can be used to obtain a currently set wrapper. - A new
is_finalizing()function can be used to check if the Python interpreter is shutting down.
- A new
- time The
monotonic()function is now always available.
(Changes to non-built-in modules will need to be documented elsewhere.)
The above list should be edited if/when progress is made on a given feature.
Discussion of Python 3.8 support
This issue is intended to track the status of Python 3.8 core features as implemented by MicroPython.
Python 3.8.0 (final) was released on the 14 October 2019. The Features for 3.8 are defined in PEP 569 and a detailed description of the changes can be found in What's New in Python 3.8.
- PEP 570, Positional-only arguments
- PEP 572, Assignment Expressions; Done, see #4908
- PEP 574, Pickle protocol 5 with out-of-band data
- PEP 578, Runtime audit hooks
- PEP 587, Python Initialization Configuration
- PEP 590, Vectorcall: a fast calling protocol for CPython
Misc
- f-strings support = for self-documenting expressions and debugging; Done, see #7649
- A
continuestatement was illegal in thefinallyclause due to a problem with the implementation. In Python 3.8 this restriction was lifted; Done, see 82c494a97e874912e7eb23d2f03f39212e343fb3 - The
bool,int, andfractions.Fractiontypes now have anas_integer_ratio()method like that found infloatanddecimal.Decimal - Constructors of
int,floatandcomplexwill now use the__index__()special method, if available and the corresponding method__int__(),__float__()or__complex__()is not available - Added support of
\N{name}escapes in regular expressions - Dict and dictviews are now iterable in reversed insertion order using
reversed() - The syntax allowed for keyword names in function calls was further restricted. In particular, f((keyword)=arg) is no longer allowed
- Generalized iterable unpacking in yield and return statements no longer requires enclosing parentheses
- When a comma is missed in code such as [(10, 20) (30, 40)], the compiler displays a SyntaxWarning with a helpful suggestion
- Arithmetic operations between subclasses of
datetime.dateordatetime.datetimeanddatetime.timedeltaobjects now return an instance of the subclass, rather than the base class - When the Python interpreter is interrupted by
Ctrl-C (SIGINT)and the resultingKeyboardInterruptexception is not caught, the Python process now exits via aSIGINTsignal or with the correct exit code such that the calling process can detect that it died due to aCtrl-C - Some advanced styles of programming require updating the
types.CodeTypeobject for an existing function - For integers, the three-argument form of the pow() function now permits the exponent to be negative in the case where the base is relatively prime to the modulus
- Dict comprehensions have been synced-up with dict literals so that the key is computed first and the value second
- The
object.__reduce__()method can now return a tuple from two to six elements long
Changes to MicroPython built-in modules
- asyncio
-
asyncio.run()has graduated from the provisional to stable API - Running
python -m asynciolaunches a natively async REPL - The exception
asyncio.CancelledErrornow inherits fromBaseExceptionrather thanExceptionand no longer inherits fromconcurrent.futures.CancelledError - Added
asyncio.Task.get_coro()for getting the wrapped coroutine within anasyncio.Task - Asyncio tasks can now be named, either by passing the name keyword argument to
asyncio.create_task()or thecreate_task()event loop method, or by calling theset_name()method on the task object - Added support for Happy Eyeballs to
asyncio.loop.create_connection(). To specify the behavior, two new parameters have been added:happy_eyeballs_delayandinterleave.
-
- gc -
get_objects()can now receive an optional generation parameter indicating a generation to get objects from- (Note, though, that while
gcis a built-in,get_objects()is not implemented for MicroPython)
- (Note, though, that while
- math
- Added new function
math.dist()for computing Euclidean distance between two points - Expanded the
math.hypot()function to handle multiple dimensions - Added new function,
math.prod(), as analogous function tosum()that returns the product of a ‘start’ value (default: 1) times an iterable of numbers - Added two new combinatoric functions
math.perm()andmath.comb() - Added a new function
math.isqrt()for computing accurate integer square roots without conversion to floating point - The function
math.factorial()no longer accepts arguments that are not int-like
- Added new function
- sys - Add new
sys.unraisablehook()function which can be overridden to control how “unraisable exceptions” are handled
(Changes to non-built-in modules will need to be documented elsewhere.)