Skip to content

Latest commit

 

History

History
1808 lines (1294 loc) · 40.9 KB

3.10.0b1.rst

File metadata and controls

1808 lines (1294 loc) · 40.9 KB

Creating :class:`sqlite3.Connection` objects now also produces sqlite3.connect and sqlite3.connect/handle :ref:`auditing events <auditing>`. Previously these events were only produced by :func:`sqlite3.connect` calls. Patch by Erlend E. Aasland.

The :mod:`ssl` module sets more secure cipher suites defaults. Ciphers without forward secrecy and with SHA-1 MAC are disabled by default. Security level 2 prohibits weak RSA, DH, and ECC keys with less than 112 bits of security. :class:`~ssl.SSLContext` defaults to minimum protocol version TLS 1.2. Settings are based on Hynek Schlawack's research.

The presence of newline or tab characters in parts of a URL could allow some forms of attacks.

Following the controlling specification for URLs defined by WHATWG :func:`urllib.parse` now removes ASCII newlines and tabs from URLs, preventing such attacks.

Ensures interpreter-level audit hooks receive the cpython.PyInterpreterState_New event when called through the _xxsubinterpreters module.

Fix invalid free in _sha3 module. The issue was introduced in 3.10.0a1. Python 3.9 and earlier are not affected.

Add audit events for :func:`sqlite3.connect/handle`, :meth:`sqlite3.Connection.enable_load_extension`, and :meth:`sqlite3.Connection.load_extension`. Patch by Erlend E. Aasland.

Add new audit event glob.glob/2 to incorporate the new root_dir and dir_fd arguments added to :func:`glob.glob` and :func:`glob.iglob`.

:mod:`ipaddress` module no longer accepts any leading zeros in IPv4 address strings. Leading zeros are ambiguous and interpreted as octal notation by some libraries. For example the legacy function :func:`socket.inet_aton` treats leading zeros as octal notation. glibc implementation of modern :func:`~socket.inet_pton` does not accept any leading zeros. For a while the :mod:`ipaddress` module used to accept ambiguous leading zeros.

Fix Regular Expression Denial of Service (ReDoS) vulnerability in :class:`urllib.request.AbstractBasicAuthHandler`. The ReDoS-vulnerable regex has quadratic worst-case complexity and it allows cause a denial of service when identifying crafted invalid RFCs. This ReDoS issue is on the client side and needs remote attackers to control the HTTP server.

Audit hooks are now fired for frame.f_code, traceback.tb_frame, and generator code/frame attribute access.

Add audit events to the :mod:`http.client` module.

Prevent classes being both a sequence and a mapping when pattern matching.

Use :c:member:`~PyTypeObject.tp_flags` on the class object to determine if the subject is a sequence or mapping when pattern matching. Avoids the need to import :mod:`collections.abc` when pattern matching.

Restore proper validation of complex literal value patterns when parsing :keyword:`!match` blocks.

Set frame.f_lineno to the line number of the 'with' kweyword when executing the call to __exit__.

If the current position in a frame has no line number then set the f_lineno attribute to None, instead of -1, to conform to PEP 626. This should not normally be possible, but might occur in some unusual circumstances.

Importing the :mod:`!_signal` module in a subinterpreter has no longer side effects.

The internal representation of line number tables is changed to not use sentinels, and an explicit length parameter is added to the out of process API function PyLineTable_InitAddressRange. This makes the handling of line number tables more robust in some circumstances.

Make :mod:`re` types immutable. Patch by Erlend E. Aasland.

Make the :class:`array.array` type immutable. Patch by Erlend E. Aasland.

Change class and module objects to lazy-create empty annotations dicts on demand. The annotations dicts are stored in the object's __dict__ for backwards compatibility.

Match patterns now use new dedicated AST nodes (MatchValue, MatchSingleton, MatchSequence, MatchStar, MatchMapping, MatchClass) rather than reusing expression AST nodes. MatchAs and MatchOr are now defined as pattern nodes rather than as expression nodes. Patch by Nick Coghlan.

Usage of await/yield/yield from and named expressions within an annotation is now forbidden when PEP 563 is activated.

When performing structural pattern matching (PEP 634), captured names are now left unbound until the entire pattern has matched successfully.

Annotations for complex targets (everything beside simple names) no longer cause any runtime effects with from __future__ import annotations.

:exc:`SyntaxError` exceptions raised by the interpreter will highlight the full error range of the expression that constitutes the syntax error itself, instead of just where the problem is detected. Patch by Pablo Galindo.

Revert making from __future__ import annotations the default. This follows the Steering Council decision to postpone PEP 563 changes to at least Python 3.11. See the original email for more information regarding the decision: https://mail.python.org/archives/list/python-dev@python.org/thread/CLVXXPQ2T2LQ5MP2Y53VVQFCXYWQJHKZ/. Patch by Pablo Galindo.

Hashes of NaN values now depend on object identity. Formerly, they always hashed to 0 even though NaN values are not equal to one another. Having the same hash for unequal values caused pile-ups in hash tables.

Improve the error message for :exc:`IndentationError` exceptions. Patch by Pablo Galindo

Constant tuple folding in bytecode optimizer now reuses tuple in constant table.

Data stack usage is much reduced for large literal and call expressions.

When printing :exc:`NameError` raised by the interpreter, :c:func:`PyErr_Display` will offer suggestions of similar variable names in the function that the exception was raised from. Patch by Pablo Galindo

Improve syntax errors for invalid dictionary literals. Patch by Pablo Galindo.

Improve syntax errors in the parser for missing commas between expressions. Patch by Pablo Galindo.

:class:`ast.alias` nodes now include source location metadata attributes e.g. lineno, col_offset.

Improve SyntaxError error messages for invalid comparisons. Patch by Pablo Galindo.

Move the flag for checking whether tracing is enabled to the C stack, from the heap. Should speed up dispatch in the interpreter.

Static methods (:func:`@staticmethod <staticmethod>`) and class methods (:func:`@classmethod <classmethod>`) now inherit the method attributes (__module__, __name__, __qualname__, __doc__, __annotations__) and have a new __wrapped__ attribute. Patch by Victor Stinner.

Fixed a bug where anext(ait, default) would erroneously return None.

:data:`~object.__match_args__` is no longer allowed to be a list.

Add GEN_START opcode. Marks start of generator, including async, or coroutine and handles sending values to a newly created generator or coroutine.

Importlib now resolves relative paths when creating module spec objects from file locations.

Static methods (:func:`@staticmethod <staticmethod>`) are now callable as regular functions. Patch by Victor Stinner.

Prevented crashes in the AST validator and optimizer when compiling some absurdly long expressions like "+0"*1000000. :exc:`RecursionError` is now raised instead.

When printing :exc:`AttributeError`, :c:func:`PyErr_Display` will offer suggestions of similar attribute names in the object that the exception was raised from. Patch by Pablo Galindo

In @dataclass(), raise a TypeError if KW_ONLY is specified more than once.

Added a total() method to collections.Counter() to compute the sum of the counts.

Change :class:`netrc.netrc` to use UTF-8 encoding before using locale encoding.

Removed an unnecessary list comprehension before looping from :func:`urllib.parse.parse_qsl`. Patch by Christoph Zwerschke and Donghee Na.

Update bundled pip to 21.1.1.

[Enum] Deprecate TypeError when non-member is used in a containment check; In 3.12 True or False will be returned instead, and containment will return True if the value is either a member of that enum or one of its members' value.

For backwards compatibility with previous minor versions of Python, if :func:`typing.get_type_hints` receives no namespace dictionary arguments, :func:`typing.get_type_hints` will search through the global then local namespaces during evaluation of stringized type annotations (string forward references) inside a class.

[Enum] Deprecate non-standard mixin format() behavior: in 3.12 the enum member, not the member's value, will be used for format() calls.

Deprecate undocumented cgi.log() API.

Fixed the :mod:`turtle` module working with non-default root window.

Update bundled pip to 21.1 and setuptools to 56.0.0

Fix a bug in the pure-Python pickle implementation when using protocol 5, where bytearray instances that occur several time in the pickled object graph would incorrectly unpickle into repeated copies of the bytearray object.

In importlib.metadata, provide a uniform interface to Description, allow for any field to be encoded with multiline values, remove continuation lines from multiline values, and add a .json property for easy access to the PEP 566 JSON-compatible form. Sync with importlib_metadata 4.0.

OpenSSL 3.0.0: :meth:`~ssl.SSLContext.load_verify_locations` now returns a consistent error message when cadata contains no valid certificate.

:mod:`urllib` can now convert Windows paths with \\?\ prefixes into URL paths.

Add :func:`inspect.get_annotations`, which safely computes the annotations defined on an object. It works around the quirks of accessing the annotations from various types of objects, and makes very few assumptions about the object passed in. :func:`inspect.get_annotations` can also correctly un-stringize stringized annotations.

:func:`inspect.signature`, :func:`inspect.from_callable`, and :func:`inspect.from_function` now call :func:`inspect.get_annotations` to retrieve annotations. This means :func:`inspect.signature` and :func:`inspect.from_callable` can now un-stringize stringized annotations, too.

platform.win32_ver derives the windows version from sys.getwindowsversion().platform_version which in turn derives the version from kernel32.dll (which can be of a different version than Windows itself). Therefore change the platform.win32_ver to determine the version using the platform module's _syscmd_ver private function to return an accurate version.

The :mod:`ssl` module now uses SSL_read_ex and SSL_write_ex internally. The functions support reading and writing of data larger than 2 GB. Writing zero-length data no longer fails with a protocol violation error.

Port _ssl extension module to multiphase initialization.

:mod:`ssl` now raises DeprecationWarning for OP_NO_SSL/TLS* options, old TLS versions, old protocols, and other features that have been deprecated since Python 3.6, 3.7, or OpenSSL 1.1.0.

PEP 612 is now implemented purely in Python; builtin types.GenericAlias objects no longer include typing.ParamSpec in __parameters__ (with the exception of collections.abc.Callable's GenericAlias). This means previously invalid uses of ParamSpec (such as list[P]) which worked in earlier versions of Python 3.10 alpha, will now raise TypeError during substitution.

The :mod:`multiprocessing` Server class now explicitly catches :exc:`SystemExit` and closes the client connection in this case. It happens when the Server.serve_client() method reaches the end of file (EOF).

Remove unused imports: pyclbr no longer uses copy, and typing no longer uses ast. Patch by Victor Stinner.

Remove an unneeded copy of the namespace passed to dataclasses.make_dataclass().

Add __iter__() method to :class:`bz2.BZ2File`, :class:`gzip.GzipFile`, and :class:`lzma.LZMAFile`. It makes iterating them about 2x faster. Patch by Inada Naoki.

Deprecate io.OpenWrapper and _pyio.OpenWrapper: use io.open and _pyio.open instead. Until Python 3.9, _pyio.open was not a static method and builtins.open was set to OpenWrapper to not become a bound method when set to a class variable. _io.open is a built-in function whereas _pyio.open is a Python function. In Python 3.10, _pyio.open() is now a static method, and builtins.open() is now io.open().

The Python :func:`!_pyio.open` function becomes a static method to behave as :func:`io.open` built-in function: don't become a bound method when stored as a class variable. It becomes possible since static methods are now callable in Python 3.10. Moreover, :func:`!_pyio.OpenWrapper` becomes a simple alias to :func:`!_pyio.open`. Patch by Victor Stinner.

Fix :exc:`KeyError` raised in :func:`typing.get_type_hints` due to synthetic modules that don't appear in sys.modules.

When :class:`subprocess.Popen` args are provided as a string or as :class:`pathlib.Path`, the Popen instance repr now shows the right thing.

[Enum] ensure exceptions raised in _missing__ are released

fix issue with enum member name matching the start of a private variable name

Fixed the return value of TypeVar.__ror__. Patch by Jelle Zijlstra.

Add match_args parameter to @dataclass decorator to allow suppression of __match_args__ generation.

OpenSSL 3.0.0: define OPENSSL_API_COMPAT 1.1.1 to suppress deprecation warnings. Python requires OpenSSL 1.1.1 APIs.

Mocks can no longer be used as the specs for other Mocks. As a result, an already-mocked object cannot have an attribute mocked using autospec=True or be the subject of a create_autospec(...) call. This can uncover bugs in tests since these Mock-derived Mocks will always pass certain tests (e.g. :func:`isinstance`) and builtin assert functions (e.g. assert_called_once_with) will unconditionally pass.

Add :const:`ssl.OP_IGNORE_UNEXPECTED_EOF` constants (OpenSSL 3.0.0)

Improve bz2.BZ2File performance by removing the RLock from BZ2File. This makes BZ2File thread unsafe in the face of multiple simultaneous readers or writers, just like its equivalent classes in :mod:`gzip` and :mod:`lzma` have always been. Patch by Inada Naoki.

OpenSSL 3.0.0: Don't call the password callback function a second time when first call has signaled an error condition.

The header files for :mod:`ssl` error codes are now OpenSSL version-specific. Exceptions will now show correct reason and library codes. The make_ssl_data.py script has been rewritten to use OpenSSL's text file with error codes.

Implement PEP 647 in the :mod:`typing` module by adding :data:`TypeGuard`.

:func:`os.path.realpath` now accepts a strict keyword-only argument. When set to True, :exc:`OSError` is raised if a path doesn't exist or a symlink loop is encountered.

In importlib.metadata, incorporate changes from importlib_metadata 3.10: Add mtime-based caching during distribution discovery. Flagged use of dict result from entry_points() as deprecated.

The P.args and P.kwargs attributes of :class:`typing.ParamSpec` are now instances of the new classes :class:`typing.ParamSpecArgs` and :class:`typing.ParamSpecKwargs`, which enables a more useful repr(). Patch by Jelle Zijlstra.

Add an encoding parameter :func:`logging.fileConfig`.

Add encoding and errors parameters to :func:`fileinput.input` and :class:`fileinput.FileInput`.

A simple_enum decorator is added to the enum module to convert a normal class into an Enum. test_simple_enum added to test simple enums against a corresponding normal Enum. Standard library modules updated to use simple_enum.

Fix an issue where :data:`~object.__match_args__` generation could fail for some :mod:`dataclasses`.

Fix :mod:`sqlite3` regression for zero-sized blobs with converters, where b"" was returned instead of None. The regression was introduced by PR 24723. Patch by Erlend E. Aasland.

:mod:`tkinter` dialog windows are now recognized as dialogs by window managers on macOS and X Window.

The following threading methods are now deprecated and should be replaced:

Patch by Jelle Zijlstra.

Deprecate find_module() and find_loader() implementations in importlib and zipimport.

:func:`turtle.textinput` and :func:`turtle.numinput` create now a transient window working on behalf of the canvas window.

Add the ability to specify keyword-only fields to dataclasses. These fields will become keyword-only arguments to the generated __init__.

Fix problem with :attr:`~ssl.SSLContext.hostname_checks_common_name`. OpenSSL does not copy hostflags from struct SSL_CTX to struct SSL.

Improve error message for :func:`tarfile.open` when :mod:`lzma` / :mod:`bz2` are unavailable. Patch by Anthony Sottile.

Allow :class:`bytes` separator argument in urllib.parse.parse_qs and urllib.parse.parse_qsl when parsing :class:`str` query strings. Previously, this raised a TypeError.

Improve :mod:`sqlite3` error handling: sqlite3_value_blob() errors that set SQLITE_NOMEM now raise :exc:`MemoryError`. Patch by Erlend E. Aasland.

New functions :func:`sysconfig.get_preferred_scheme` and :func:`sysconfig.get_default_scheme` are added to query a platform for its preferred "user", "home", and "prefix" (default) scheme names.

Improve :meth:`sqlite3.Connection.backup` error handling. The error message for non-existent target database names is now unknown database <database name> instead of SQL logic error. Patch by Erlend E. Aasland.

Install schemes in distutils.command.install are now loaded from :mod:`sysconfig`.

distutils.sysconfig has been merged to :mod:`sysconfig`.

Fixed processing of a dataclass that inherits from a frozen dataclass with no fields. It is now correctly detected as an error.

:mod:`pprint` now has support for :class:`dataclasses.dataclass`. Patch by Lewis Gaul.

Add pathlib.Path.hardlink_to() method that supersedes link_to(). The new method has the same argument order as symlink_to().

:func:`typing.get_type_hints` now checks the local namespace of a class when evaluating PEP 563 annotations inside said class.

Add slots parameter to dataclasses.dataclass decorator to automatically generate __slots__ for class. Patch provided by Yurii Karabas.

Deprecated use of :func:`asyncio.get_event_loop` without running event loop. Emit deprecation warning for :mod:`asyncio` functions which implicitly create a :class:`~asyncio.Future` or :class:`~asyncio.Task` objects if there is no running event loop and no explicit loop argument is passed: :func:`~asyncio.ensure_future`, :func:`~asyncio.wrap_future`, :func:`~asyncio.gather`, :func:`~asyncio.shield`, :func:`~asyncio.as_completed` and constructors of :class:`~asyncio.Future`, :class:`~asyncio.Task`, :class:`~asyncio.StreamReader`, :class:`~asyncio.StreamReaderProtocol`.

Certificate and PrivateKey classes were added to the ssl module. Certificates and keys can now be loaded from memory buffer, too.

Use a new output buffer management code for :mod:`bz2` / :mod:`lzma` / :mod:`zlib` modules, and add .readall() function to _compression.DecompressReader class. These bring some performance improvements. Patch by Ma Lin.

The :func:`ssl.get_server_certificate` function now has a timeout parameter.

Fix thread locks in zlib module may go wrong in rare case. Patch by Ma Lin.

Fix dataclasses with InitVars and :func:`~dataclasses.replace`. Patch by Claudiu Popa.

Expose X509_V_FLAG_PARTIAL_CHAIN ssl flag

:func:`ssl.RAND_status` now returns a boolean value (as documented) instead of 1 or 0.

:meth:`pathlib.Path.stat` and :meth:`~pathlib.Path.chmod` now accept a follow_symlinks keyword-only argument for consistency with corresponding functions in the :mod:`os` module.

:func:`os.path.expanduser` now refuses to guess Windows home directories if the basename of current user's home directory does not match their username.

:meth:`pathlib.Path.expanduser` and :meth:`~pathlib.Path.home` now consistently raise :exc:`RuntimeError` exception when a home directory cannot be resolved. Previously a :exc:`KeyError` exception could be raised on Windows when the "USERNAME" environment variable was unset.

Added SNI support to :func:`ssl.get_server_certificate`.

Covariance, Pearson's correlation, and simple linear regression functionality was added to statistics module. Patch by Tymoteusz Wołodźko.

Provide a locale.localize() function, which converts a normalized number string into a locale format.

Fix a regression in the handling of ctypes' :data:`ctypes.c_wchar_p` type: embedded null characters would cause a :exc:`ValueError` to be raised. Patch by Zackery Spytz.

Add "Annotations Best Practices" document as a new HOWTO.

Document the new :c:macro:`Py_TPFLAGS_MAPPING` and :c:macro:`Py_TPFLAGS_SEQUENCE` type flags.

The documentation on the PyContextVar C-API was clarified.

Update dataclasses documentation to express that FrozenInstanceError is derived from AttributeError.

Fix the Sphinx glossary_search extension: create the _static/ sub-directory if it doesn't exist.

Update documentation to reflect that unparenthesized lambda expressions can no longer be the expression part in an if clause in comprehensions and generator expressions since Python 3.9.

Fixing the example code in Doc/extending/extending.rst to declare and initialize the pmodule variable to be of the right type.

Fix test_logging.test_namer_rotator_inheritance() on Windows: use :func:`os.replace` rather than :func:`os.rename`. Patch by Victor Stinner.

Fix a race condition in the SMTP test of test_logging. Don't close a file descriptor (socket) from a different thread while asyncore.loop() is polling the file descriptor. Patch by Victor Stinner.

:mod:`test.libregrtest` now marks a test as ENV_CHANGED (altered the execution environment) if a thread raises an exception but does not catch it. It sets a hook on :func:`threading.excepthook`. Use --fail-env-changed option to mark the test as failed. Patch by Victor Stinner.

Tests multiple OpenSSL versions on GitHub Actions. Use ccache to speed up testing.

OpenSSL 3.0.0: Disable testing of legacy protocols TLS 1.0 and 1.1. Tests are failing with TLSV1_ALERT_INTERNAL_ERROR.

Improved generated code refresh (AST/tokens/opcodes/keywords) on Windows.

Implement PEP 644. Python now requires OpenSSL 1.1.1 or newer.

Adds additional arguments to :func:`os.startfile` function.

Avoid raising errors from :meth:`pathlib.Path.exists` when passed an invalid filename.

Fixed :func:`os.stat` failing on inaccessible directories with a trailing slash, rather than falling back to the parent directory's metadata. This implicitly affected :func:`os.path.exists` and :func:`os.path.isdir`.

Fixed decoding of host names in :func:`socket.gethostbyaddr` and :func:`socket.gethostbyname_ex`.

Updated pegen regeneration script on Windows to find and use Python 3.8 or higher. Prior to this, pegen regeneration already required 3.8 or higher, but the script may have used lower versions of Python.

Actually updates Windows release to OpenSSL 1.1.1k. Earlier releases were mislabelled and actually included 1.1.1i again.

Update Tcl and Tk to 8.6.11 in Windows installer.

Upgrade Windows installer to use SQLite 3.35.5.

Fix WindowsConsoleIO errors in the presence of fd redirection. Patch by Segev Finer.

Fix check for macOS SDK paths when building Python. Narrow search to match contents of SDKs, namely only files in /System/Library, /System/IOSSupport, and /usr other than /usr/local. Previously, anything under /System was assumed to be in an SDK which causes problems with the new file system layout in 10.15+ where user file systems may appear to be mounted under /System. Paths in /Library were also incorrectly treated as SDK locations.

Drop support for MACOSX_DEPLOYMENT_TARGET < 10.3

Provide "python3.x-intel64" executable to allow reliably forcing macOS universal2 framework builds to run under Rosetta 2 Intel-64 emulation on Apple Silicon Macs. This can be useful for testing or when universal2 wheels are not yet available.

Build SQLite with SQLITE_OMIT_AUTOINIT on macOS. Patch by Erlend E. Aasland.

Update macOS installer to use SQLite 3.35.4.

Mac/BuildScript/build-installer.py will now use "--enable-optimizations" and --with-lto when building on macOS 10.15 or later.

Add mouse actions to the shell sidebar. Left click and optional drag selects one or more lines, as with the editor line number sidebar. Right click after selecting raises a context menu with 'copy with prompts'. This zips together prompts from the sidebar with lines from the selected text.

Fix reference leak in test_sidebar and test_squeezer. Patches by Terry Jan Reedy and Pablo Galindo

Indent IDLE Shell input with spaces instead of tabs

IDLE dialog windows are now recognized as dialogs by window managers on macOS and X Window.

IDLE's shell now shows prompts in a separate side-bar.

Add a new :c:macro:`Py_TPFLAGS_DISALLOW_INSTANTIATION` type flag to disallow creating type instances. Patch by Victor Stinner.

Remove the now unused PYMALLOC_DEBUG macro. Debug hooks on memory allocators are now installed by default if Python is built in debug mode (if Py_DEBUG macro is defined). Moreover, they can now be used on Python build in release mode (ex: using PYTHONMALLOC=debug environment variable).

_PyInterpreterState_IDIncref() now calls _PyInterpreterState_IDInitref() and always increments id_refcount. Previously, calling _xxsubinterpreters.get_current() could create an id_refcount inconsistency when a _xxsubinterpreters.InterpreterID object was deallocated. Patch by Victor Stinner.

Add new C-API functions to control the state of the garbage collector: :c:func:`PyGC_Enable()`, :c:func:`PyGC_Disable()`, :c:func:`PyGC_IsEnabled()`, corresponding to the functions in the :mod:`gc` module.

Introduce :c:macro:`Py_TPFLAGS_IMMUTABLETYPE` flag for immutable type objects, and modify :c:func:`PyType_Ready` to set it for static types. Patch by Erlend E. Aasland.

:c:func:`PyMem_Calloc` is now available in the limited C API (Py_LIMITED_API).

:c:func:`PyOS_ReadlineFunctionPointer` is no longer exported by limited C API headers and by python3.dll on Windows. Like any function that takes FILE*, it is not part of the stable ABI.

Stable ABI and limited API definitions are generated from a central manifest (PEP 652).

Add the :c:func:`Py_Is(x, y) <Py_Is>` function to test if the x object is the y object, the same as x is y in Python. Add also the :c:func:`Py_IsNone`, :c:func:`Py_IsTrue`, :c:func:`Py_IsFalse` functions to test if an object is, respectively, the None singleton, the True singleton or the False singleton. Patch by Victor Stinner.