← index #2415Issue #1329
Related · high · value 0.075
QUERY · ISSUE

Discussion of Python 3.6 support

openby dpgeorgeopened 2016-09-13updated 2022-01-22
rfc

Python 3.6 beta 1 was released on 12 Sep 2016, and a summary of the new features can be found here: https://docs.python.org/3.6/whatsnew/3.6.html . Being in beta stage means we can provide feedback on any of the new features/changes if we like.

(See #1329 for Python 3.5 support, which is still pending.)

New syntax features:

  • PEP 498 - Literal String Formatting : f-strings were implemented in 692d36d779192f32371f7f9daa845b566f26968d
  • PEP 515 - Underscores in Numeric Literals; done in 6a445b60fad87c94a1d00ce3a043b881a1f7a5a4 ; see also #3831
  • PEP 525 - Asynchronous Generators (provisional); see #6668
  • PEP 526 - Syntax for Variable Annotations (provisional) for uPy: requires changes to parser, relevant for viper; see #6126
  • PEP 530 - Asynchronous Comprehensions

New built-in features:

  • PEP 468 - Preserving the order of **kwargs in a function
  • PEP 487 - Simpler customization of class creation
  • PEP 520 - Preserving Class Attribute Definition Order

Standard library changes:

  • PEP 495 - Local Time Disambiguation
  • PEP 506 - Adding A Secrets Module To The Standard Library [not relevant to uPy core]
  • PEP 519 - Adding a file system path protocol

CPython internals (not exposed at the Python level so not relevant to uPy):

  • PEP 509 - Add a private version to dict
  • PEP 523 - Adding a frame evaluation API to CPython

Linux/Windows changes:

  • PEP 524 - Make os.urandom() blocking on Linux (during system startup)
  • PEP 528 - Change Windows console encoding to UTF-8 (provisional)
  • PEP 529 - Change Windows filesystem encoding to UTF-8 (provisional)

Other Language Changes

  • A global or nonlocal statement must now textually appear before the first use of the affected name in the same scope. Previously this was a SyntaxWarning.
  • It is now possible to set a special method to None to indicate that the corresponding operation is not available. For example, if a class sets __iter__() to None, the class is not iterable.
  • Long sequences of repeated traceback lines are now abbreviated as "[Previous line repeated {count} more times]"
  • Import now raises the new exception ModuleNotFoundError when it cannot find a module. Code that currently checks for ImportError (in try-except) will still work.
  • Class methods relying on zero-argument super() will now work correctly when called from metaclass methods during class creation.

Changes to MicroPython built-in modules

  • array - Exhausted iterators of array.array will now stay exhausted even if the iterated array is extended.
  • asyncio (many, may need another ticket)
  • binascii - The b2a_base64() function now accepts an optional newline keyword argument to control whether the newline character is appended to the return value.
  • cmath
    • The new cmath.tau (τ) constant has been added.
    • New constants: cmath.inf and cmath.nan to match math.inf and math.nan, and also cmath.infj and cmath.nanj to match the format used by complex repr.
  • collections
    • The new Collection abstract base class has been added to represent sized iterable container classes.
    • The new Reversible abstract base class represents iterable classes that also provide the __reversed__() method.
    • The new AsyncGenerator abstract base class represents asynchronous generators
    • The namedtuple() function now accepts an optional keyword argument module, which, when specified, is used for the __module__ attribute of the returned named tuple class.
    • The verbose and rename arguments for namedtuple() are now keyword-only.
    • Recursive collections.deque instances can now be pickled.
  • hashlib
    • hashlib supports OpenSSL 1.1.0. The minimum recommend version is 1.0.2.
    • BLAKE2 hash functions were added to the module. blake2b() and blake2s() are always available and support the full feature set of BLAKE2.
    • The SHA-3 hash functions sha3_224(), sha3_256(), sha3_384(), sha3_512(), and SHAKE hash functions shake_128() and shake_256() were added.
    • The password-based key derivation function scrypt() is now available with OpenSSL 1.1.0 and newer.
  • json - json.load() and json.loads() now support binary input. Encoded JSON should be represented using either UTF-8, UTF-16, or UTF-32.
  • math - The tau (τ) constant has been added to the math and cmath modules.
  • os
    • See the summary of PEP 519 for details on how the os and os.path modules now support path-like objects.
    • A new close() method allows explicitly closing a scandir() iterator. The scandir() iterator now supports the context manager protocol.
    • On Linux, os.urandom() now blocks until the system urandom entropy pool is initialized to increase the security.
    • The Linux getrandom() syscall (get random bytes) is now exposed as the new os.getrandom() function.
  • re
    • Added support of modifier spans in regular expressions. Examples: '(?i:p)ython' matches 'python' and 'Python', but not 'PYTHON'; '(?i)g(?-i:v)r' matches 'GvR' and 'gvr', but not 'GVR'.
    • Match object groups can be accessed by __getitem__, which is equivalent to group(). So mo['name'] is now equivalent to mo.group('name').
    • Match objects now support index-like objects as group indices.
  • socket
    • The ioctl() function now supports the SIO_LOOPBACK_FAST_PATH control code.
    • The getsockopt() constants SO_DOMAIN, SO_PROTOCOL, SO_PEERSEC, and SO_PASSSEC are now supported.
    • The setsockopt() now supports the setsockopt(level, optname, None, optlen: int) form.
    • The socket module now supports the address family AF_ALG to interface with Linux Kernel crypto API. ALG_*, SOL_ALG and sendmsg_afalg() were added.
    • New Linux constants TCP_USER_TIMEOUT and TCP_CONGESTION were added.
  • ssl
    • ssl supports OpenSSL 1.1.0. The minimum recommend version is 1.0.2.
    • 3DES has been removed from the default cipher suites and ChaCha20 Poly1305 cipher suites have been added.
    • SSLContext has better default configuration for options and ciphers.
    • SSL session can be copied from one client-side connection to another with the new SSLSession class. TLS session resumption can speed up the initial handshake, reduce latency and improve performance.
    • The new get_ciphers() method can be used to get a list of enabled ciphers in order of cipher priority.
    • All constants and flags have been converted to IntEnum and IntFlags.
    • Server and client-side specific TLS protocols for SSLContext were added.
    • Added SSLContext.post_handshake_auth to enable and ssl.SSLSocket.verify_client_post_handshake() to initiate TLS 1.3 post-handshake authentication.
  • struct - now supports IEEE 754 half-precision floats via the 'e' format specifier.
  • sys - The new getfilesystemencodeerrors() function returns the name of the error mode used to convert between Unicode filenames and bytes filenames.
  • zlib - The compress() and decompress() functions now accept keyword arguments.

(Changes to non-built-in modules will need to be documented elsewhere.)

CANDIDATE · ISSUE

Discussion of Python 3.5 support

openby dpgeorgeopened 2015-06-15updated 2022-04-02
rfc

MicroPython aims to implement the Python 3.x "standard". At the moment x is currently 4, ie we try to be compatible with CPython 3.4. It's new territory as to what to do when CPython evolves to larger version numbers. I would say we should try to follow the changes and implement them when possible/sensible.

Python 3.5 had a feature freeze on 24 May 2015 and is scheduled for final release on 13 September 2015. This ticket here is about discussing if, what and how we should upgrade uPy to version 3.5 of the language. It may be that some issues need to break off into separate tickets and that's fine but we should link to them from this one.

The PEP discussing the new features in 3.5: https://www.python.org/dev/peps/pep-0478/

A more friendly overview in the docs: https://docs.python.org/3.5/whatsnew/3.5.html

Below is a list of finalised/accepted PEPs for 3.5 grouped into their impact to MicroPython.

Extensions to the syntax:

  • PEP 448 - additional unpacking generalizations; see #5807
  • PEP 465 - a new matrix multiplication operator; done in 2069c563f9e944d8f45e524b425fff23208e8153
  • PEP 492 - coroutines with async and await syntax

Extensions and changes to the runtime:

  • PEP 461 - %-formatting for binary strings - tentatively done
  • PEP 475 - retrying system calls that fail with EINTR - done in 9418611c8aa860812e28b6e2b6bfde7be5817b43
  • PEP 479 - change StopIteration handling inside generators - done in 3f6ffe059f64b3ebc44dc0bbc63452cb8850702b

Standard library changes:

  • PEP 471 - os.scandir()
  • PEP 485 - math.isclose(), a function for testing approximate equality - done in af5c998f37ddc62abfd36e0b8be511c392fc25d8

Miscellaneous changes that are not relevant to MicroPython:

  • PEP 441 - improved Python zip application support
  • PEP 486 - make the Python Launcher aware of virtual environments
  • PEP 484 - type hints (advisory only)
  • PEP 488 - elimination of PYO files
  • PEP 489 - redesigning extension module loading

Other Language Changes

  • Added the "namereplace" error handlers. The "backslashreplace" error handlers now work with decoding and translating.
  • Property docstrings are now writable. This is especially useful for collections.namedtuple() docstrings.
  • Circular imports involving relative imports are now supported.

New modules

Changes to MicroPython built-in modules

  • asyncio (many, may need another ticket)
  • cmath - A new function isclose() provides a way to test for approximate equality.
  • collections
    • The OrderedDict class is now implemented in C, which makes it 4 to 100 times faster.
    • OrderedDict.items(), OrderedDict.keys(), OrderedDict.values() views now support reversed() iteration.
    • The deque class now defines index(), insert(), and copy(), and supports the + and * operators.
    • Docstrings produced by namedtuple() can now be updated.
    • The UserString class now implements the __getnewargs__(), __rmod__(), casefold(), format_map(), isprintable(), and maketrans() methods to match the corresponding methods of str.
  • heapq - Element comparison in merge() can now be customized by passing a key function in a new optional key keyword argument, and a new optional reverse keyword argument can be used to reverse element comparison
  • io - A new BufferedIOBase.readinto1() method, that uses at most one call to the underlying raw stream's RawIOBase.read() or RawIOBase.readinto() methods.
  • json - JSON decoder now raises JSONDecodeError instead of ValueError to provide better context information about the error.
  • math
    • Two new constants have been added to the math module: inf and nan.
    • A new function isclose() provides a way to test for approximate equality.
    • A new gcd() function has been added. The fractions.gcd() function is now deprecated.
  • os
    • The new scandir() function returning an iterator of DirEntry objects has been added.
    • The urandom() function now uses the getrandom() syscall on Linux 3.17 or newer, and getentropy() on OpenBSD 5.6 and newer, removing the need to use /dev/urandom and avoiding failures due to potential file descriptor exhaustion.
    • New get_blocking() and set_blocking() functions allow getting and setting a file descriptor's blocking mode (O_NONBLOCK.)
    • There is a new os.path.commonpath() function returning the longest common sub-path of each passed pathname.
  • re
    • References and conditional references to groups with fixed length are now allowed in lookbehind assertions.
    • The number of capturing groups in regular expressions is no longer limited to 100.
    • The sub() and subn() functions now replace unmatched groups with empty strings instead of raising an exception.
    • The re.error exceptions have new attributes, msg, pattern, pos, lineno, and colno, that provide better context information about the error
  • socket
    • Functions with timeouts now use a monotonic clock, instead of a system clock.
    • A new socket.sendfile() method allows sending a file over a socket by using the high-performance os.sendfile() function on UNIX, resulting in uploads being from 2 to 3 times faster than when using plain socket.send().
    • The socket.sendall() method no longer resets the socket timeout every time bytes are received or sent. The socket timeout is now the maximum total duration to send all data.
    • The backlog argument of the socket.listen() method is now optional. By default it is set to SOMAXCONN or to 128, whichever is less.
  • ssl
    • Memory BIO Support
    • Application-Layer Protocol Negotiation Support
    • There is a new SSLSocket.version() method to query the actual protocol version in use.
    • The SSLSocket class now implements a SSLSocket.sendfile() method.
    • The SSLSocket.send() method now raises either the ssl.SSLWantReadError or ssl.SSLWantWriteError exception on a non-blocking socket if the operation would block. Previously, it would return 0.
    • The cert_time_to_seconds() function now interprets the input time as UTC and not as local time, per RFC 5280. Additionally, the return value is always an int.
    • New SSLObject.shared_ciphers() and SSLSocket.shared_ciphers() methods return the list of ciphers sent by the client during the handshake.
    • The SSLSocket.do_handshake(), SSLSocket.read(), SSLSocket.shutdown(), and SSLSocket.write() methods of the SSLSocket class no longer reset the socket timeout every time bytes are received or sent.
    • The match_hostname() function now supports matching of IP addresses.
  • sys
    • A new set_coroutine_wrapper() function allows setting a global hook that will be called whenever a coroutine object is created by an async def function. A corresponding get_coroutine_wrapper() can be used to obtain a currently set wrapper.
    • A new is_finalizing() function can be used to check if the Python interpreter is shutting down.
  • time The monotonic() function is now always available.

(Changes to non-built-in modules will need to be documented elsewhere.)

The above list should be edited if/when progress is made on a given feature.

Keyboard

j / / n
next pair
k / / p
previous pair
1 / / h
show query pane
2 / / l
show candidate pane
c
copy suggested comment
r
toggle reasoning
g i
go to index
?
show this help
esc
close overlays

press ? or esc to close

copied