From 7f3d7f8f10938618bcf8648d92f6a3da0a01117f Mon Sep 17 00:00:00 2001 From: Jukka Lehtosalo Date: Wed, 25 Sep 2024 14:49:22 +0100 Subject: [PATCH] [PEP 695] Further documentation updates (#17826) Finish work started in #17816. Document `type` statement when discussing type aliases. Update some examples to have both old-style and new-style variants. In less common scenarios, examples only use a single syntax variant to reduce verbosity. Also update some examples to generally use more modern features. Closes #17810. --- docs/source/additional_features.rst | 18 +++----- docs/source/cheat_sheet_py3.rst | 15 ++++++- docs/source/error_code_list.rst | 20 +++------ docs/source/kinds_of_types.rst | 47 +++++++++++++++----- docs/source/literal_types.rst | 10 ++--- docs/source/metaclasses.rst | 8 ++-- docs/source/more_types.rst | 67 +++++++++++++++++++---------- docs/source/protocols.rst | 9 ++-- docs/source/runtime_troubles.rst | 14 +++--- docs/source/type_narrowing.rst | 14 ++---- 10 files changed, 135 insertions(+), 87 deletions(-) diff --git a/docs/source/additional_features.rst b/docs/source/additional_features.rst index ae625c157654..a9c3177d32a2 100644 --- a/docs/source/additional_features.rst +++ b/docs/source/additional_features.rst @@ -46,21 +46,18 @@ define dataclasses. For example: UnorderedPoint(1, 2) < UnorderedPoint(3, 4) # Error: Unsupported operand types Dataclasses can be generic and can be used in any other way a normal -class can be used: +class can be used (Python 3.12 syntax): .. code-block:: python from dataclasses import dataclass - from typing import Generic, TypeVar - - T = TypeVar('T') @dataclass - class BoxedData(Generic[T]): + class BoxedData[T]: data: T label: str - def unbox(bd: BoxedData[T]) -> T: + def unbox[T](bd: BoxedData[T]) -> T: ... val = unbox(BoxedData(42, "")) # OK, inferred type is int @@ -98,17 +95,16 @@ does **not** work: To have Mypy recognize a wrapper of :py:func:`dataclasses.dataclass ` -as a dataclass decorator, consider using the :py:func:`~typing.dataclass_transform` decorator: +as a dataclass decorator, consider using the :py:func:`~typing.dataclass_transform` +decorator (example uses Python 3.12 syntax): .. code-block:: python from dataclasses import dataclass, Field - from typing import TypeVar, dataclass_transform - - T = TypeVar('T') + from typing import dataclass_transform @dataclass_transform(field_specifiers=(Field,)) - def my_dataclass(cls: type[T]) -> type[T]: + def my_dataclass[T](cls: type[T]) -> type[T]: ... return dataclass(cls) diff --git a/docs/source/cheat_sheet_py3.rst b/docs/source/cheat_sheet_py3.rst index b8e43960fd09..ca6dfc8eb63a 100644 --- a/docs/source/cheat_sheet_py3.rst +++ b/docs/source/cheat_sheet_py3.rst @@ -349,7 +349,20 @@ Decorators ********** Decorator functions can be expressed via generics. See -:ref:`declaring-decorators` for more details. +:ref:`declaring-decorators` for more details. Example using Python 3.12 +syntax: + +.. code-block:: python + + from typing import Any, Callable + + def bare_decorator[F: Callable[..., Any]](func: F) -> F: + ... + + def decorator_args[F: Callable[..., Any]](url: str) -> Callable[[F], F]: + ... + +The same example using pre-3.12 syntax: .. code-block:: python diff --git a/docs/source/error_code_list.rst b/docs/source/error_code_list.rst index ad73bc999f00..ec069fdcba1d 100644 --- a/docs/source/error_code_list.rst +++ b/docs/source/error_code_list.rst @@ -434,15 +434,11 @@ Check type variable values [type-var] Mypy checks that value of a type variable is compatible with a value restriction or the upper bound type. -Example: +Example (Python 3.12 syntax): .. code-block:: python - from typing import TypeVar - - T1 = TypeVar('T1', int, float) - - def add(x: T1, y: T1) -> T1: + def add[T1: (int, float)](x: T1, y: T1) -> T1: return x + y add(4, 5.5) # OK @@ -783,27 +779,25 @@ Example: Safe handling of abstract type object types [type-abstract] ----------------------------------------------------------- -Mypy always allows instantiating (calling) type objects typed as ``Type[t]``, +Mypy always allows instantiating (calling) type objects typed as ``type[t]``, even if it is not known that ``t`` is non-abstract, since it is a common pattern to create functions that act as object factories (custom constructors). Therefore, to prevent issues described in the above section, when an abstract -type object is passed where ``Type[t]`` is expected, mypy will give an error. -Example: +type object is passed where ``type[t]`` is expected, mypy will give an error. +Example (Python 3.12 syntax): .. code-block:: python from abc import ABCMeta, abstractmethod - from typing import List, Type, TypeVar class Config(metaclass=ABCMeta): @abstractmethod def get_value(self, attr: str) -> str: ... - T = TypeVar("T") - def make_many(typ: Type[T], n: int) -> List[T]: + def make_many[T](typ: type[T], n: int) -> list[T]: return [typ() for _ in range(n)] # This will raise if typ is abstract - # Error: Only concrete class can be given where "Type[Config]" is expected [type-abstract] + # Error: Only concrete class can be given where "type[Config]" is expected [type-abstract] make_many(Config, 5) .. _code-safe-super: diff --git a/docs/source/kinds_of_types.rst b/docs/source/kinds_of_types.rst index d07eb40753f3..e645a27095d1 100644 --- a/docs/source/kinds_of_types.rst +++ b/docs/source/kinds_of_types.rst @@ -434,19 +434,20 @@ the runtime with some limitations (see :ref:`runtime_troubles`). Type aliases ************ -In certain situations, type names may end up being long and painful to type: +In certain situations, type names may end up being long and painful to type, +especially if they are used frequently: .. code-block:: python - def f() -> Union[list[dict[tuple[int, str], set[int]]], tuple[str, list[str]]]: + def f() -> list[dict[tuple[int, str], set[int]]] | tuple[str, list[str]]: ... When cases like this arise, you can define a type alias by simply -assigning the type to a variable: +assigning the type to a variable (this is an *implicit type alias*): .. code-block:: python - AliasType = Union[list[dict[tuple[int, str], set[int]]], tuple[str, list[str]]] + AliasType = list[dict[tuple[int, str], set[int]]] | tuple[str, list[str]] # Now we can use AliasType in place of the full name: @@ -459,8 +460,18 @@ assigning the type to a variable: another type -- it's equivalent to the target type except for :ref:`generic aliases `. -Since Mypy 0.930 you can also use *explicit type aliases*, which were -introduced in :pep:`613`. +Python 3.12 introduced the ``type`` statement for defining *explicit type aliases*. +Explicit type aliases are unambiguous and can also improve readability by +making the intent clear: + +.. code-block:: python + + type AliasType = list[dict[tuple[int, str], set[int]]] | tuple[str, list[str]] + + # Now we can use AliasType in place of the full name: + + def f() -> AliasType: + ... There can be confusion about exactly when an assignment defines an implicit type alias -- for example, when the alias contains forward references, invalid types, or violates some other @@ -469,8 +480,17 @@ distinction between an unannotated variable and a type alias is implicit, ambiguous or incorrect type alias declarations default to defining a normal variable instead of a type alias. -Explicit type aliases are unambiguous and can also improve readability by -making the intent clear: +Aliases defined using the ``type`` statement have these properties, which +distinguish them from implicit type aliases: + +* The definition may contain forward references without having to use string + literal escaping, since it is evaluated lazily. +* The alias can be used in type annotations, type arguments, and casts, but + it can't be used in contexts which require a class object. For example, it's + not valid as a base class and it can't be used to construct instances. + +There is also use an older syntax for defining explicit type aliases, which was +introduced in Python 3.10 (:pep:`613`): .. code-block:: python @@ -604,14 +624,21 @@ doesn't see that the ``buyer`` variable has type ``ProUser``: buyer.pay() # Rejected, not a method on User However, using the ``type[C]`` syntax and a type variable with an upper bound (see -:ref:`type-variable-upper-bound`) we can do better: +:ref:`type-variable-upper-bound`) we can do better (Python 3.12 syntax): + +.. code-block:: python + + def new_user[U: User](user_class: type[U]) -> U: + # Same implementation as before + +Here is the example using the legacy syntax (Python 3.11 and earlier): .. code-block:: python U = TypeVar('U', bound=User) def new_user(user_class: type[U]) -> U: - # Same implementation as before + # Same implementation as before Now mypy will infer the correct type of the result when we call ``new_user()`` with a specific subclass of ``User``: diff --git a/docs/source/literal_types.rst b/docs/source/literal_types.rst index 283bf7f9dba1..972fce72740f 100644 --- a/docs/source/literal_types.rst +++ b/docs/source/literal_types.rst @@ -264,19 +264,15 @@ use the same technique with regular objects, tuples, or namedtuples. Similarly, tags do not need to be specifically str Literals: they can be any type you can normally narrow within ``if`` statements and the like. For example, you could have your tags be int or Enum Literals or even regular classes you narrow -using ``isinstance()``: +using ``isinstance()`` (Python 3.12 syntax): .. code-block:: python - from typing import Generic, TypeVar, Union - - T = TypeVar('T') - - class Wrapper(Generic[T]): + class Wrapper[T]: def __init__(self, inner: T) -> None: self.inner = inner - def process(w: Union[Wrapper[int], Wrapper[str]]) -> None: + def process(w: Wrapper[int] | Wrapper[str]) -> None: # Doing `if isinstance(w, Wrapper[int])` does not work: isinstance requires # that the second argument always be an *erased* type, with no generics. # This is because generics are a typing-only concept and do not exist at diff --git a/docs/source/metaclasses.rst b/docs/source/metaclasses.rst index 396d7dbb42cc..a3ee25f16054 100644 --- a/docs/source/metaclasses.rst +++ b/docs/source/metaclasses.rst @@ -34,13 +34,12 @@ Mypy supports the lookup of attributes in the metaclass: .. code-block:: python - from typing import Type, TypeVar, ClassVar - T = TypeVar('T') + from typing import ClassVar, Self class M(type): count: ClassVar[int] = 0 - def make(cls: Type[T]) -> T: + def make(cls) -> Self: M.count += 1 return cls() @@ -56,6 +55,9 @@ Mypy supports the lookup of attributes in the metaclass: b: B = B.make() # metaclasses are inherited print(B.count + " objects were created") # Error: Unsupported operand types for + ("int" and "str") +.. note:: + In Python 3.10 and earlier, ``Self`` is available in ``typing_extensions``. + .. _limitations: Gotchas and limitations of metaclass support diff --git a/docs/source/more_types.rst b/docs/source/more_types.rst index cb3ef64b39a7..2d3ce5925c02 100644 --- a/docs/source/more_types.rst +++ b/docs/source/more_types.rst @@ -256,11 +256,34 @@ method receives an integer we return a single item. If it receives a ``slice``, we return a :py:class:`~typing.Sequence` of items. We can precisely encode this relationship between the argument and the -return type by using overloads like so: +return type by using overloads like so (Python 3.12 syntax): .. code-block:: python - from typing import Sequence, TypeVar, Union, overload + from collections.abc import Sequence + from typing import overload + + class MyList[T](Sequence[T]): + @overload + def __getitem__(self, index: int) -> T: ... + + @overload + def __getitem__(self, index: slice) -> Sequence[T]: ... + + def __getitem__(self, index: int | slice) -> T | Sequence[T]: + if isinstance(index, int): + # Return a T here + elif isinstance(index, slice): + # Return a sequence of Ts here + else: + raise TypeError(...) + +Here is the same example using the legacy syntax (Python 3.11 and earlier): + +.. code-block:: python + + from collections.abc import Sequence + from typing import TypeVar, Union, overload T = TypeVar('T') @@ -697,14 +720,13 @@ Restricted methods in generic classes ------------------------------------- In generic classes some methods may be allowed to be called only -for certain values of type arguments: +for certain values of type arguments (Python 3.12 syntax): .. code-block:: python - T = TypeVar('T') - - class Tag(Generic[T]): + class Tag[T]: item: T + def uppercase_item(self: Tag[str]) -> str: return self.item.upper() @@ -714,18 +736,18 @@ for certain values of type arguments: ts.uppercase_item() # This is OK This pattern also allows matching on nested types in situations where the type -argument is itself generic: +argument is itself generic (Python 3.12 syntax): .. code-block:: python - T = TypeVar('T', covariant=True) - S = TypeVar('S') + from collections.abc import Sequence - class Storage(Generic[T]): + class Storage[T]: def __init__(self, content: T) -> None: - self.content = content - def first_chunk(self: Storage[Sequence[S]]) -> S: - return self.content[0] + self._content = content + + def first_chunk[S](self: Storage[Sequence[S]]) -> S: + return self._content[0] page: Storage[list[str]] page.first_chunk() # OK, type is "str" @@ -734,13 +756,13 @@ argument is itself generic: # "first_chunk" with type "Callable[[Storage[Sequence[S]]], S]" Finally, one can use overloads on self-type to express precise types of -some tricky methods: +some tricky methods (Python 3.12 syntax): .. code-block:: python - T = TypeVar('T') + from typing import overload, Callable - class Tag(Generic[T]): + class Tag[T]: @overload def export(self: Tag[str]) -> str: ... @overload @@ -799,23 +821,22 @@ Precise typing of alternative constructors ------------------------------------------ Some classes may define alternative constructors. If these -classes are generic, self-type allows giving them precise signatures: +classes are generic, self-type allows giving them precise +signatures (Python 3.12 syntax): .. code-block:: python - T = TypeVar('T') - - class Base(Generic[T]): - Q = TypeVar('Q', bound='Base[T]') + from typing import Self + class Base[T]: def __init__(self, item: T) -> None: self.item = item @classmethod - def make_pair(cls: Type[Q], item: T) -> tuple[Q, Q]: + def make_pair(cls, item: T) -> tuple[Self, Self]: return cls(item), cls(item) - class Sub(Base[T]): + class Sub[T](Base[T]): ... pair = Sub.make_pair('yes') # Type is "tuple[Sub[str], Sub[str]]" diff --git a/docs/source/protocols.rst b/docs/source/protocols.rst index 731562867691..e143808e6c25 100644 --- a/docs/source/protocols.rst +++ b/docs/source/protocols.rst @@ -225,7 +225,7 @@ such as trees and linked lists: .. code-block:: python - from typing import TypeVar, Optional, Protocol + from typing import Optional, Protocol class TreeLike(Protocol): value: int @@ -315,8 +315,8 @@ member: # different name and kind in the callback Callback protocols and :py:data:`~typing.Callable` types can be used mostly interchangeably. -Argument names in :py:meth:`__call__ ` methods must be identical, unless -a double underscore prefix is used. For example: +Parameter names in :py:meth:`__call__ ` methods must be identical, unless +the parameters are positional-only. Example (using the legacy syntax for generic functions): .. code-block:: python @@ -325,7 +325,8 @@ a double underscore prefix is used. For example: T = TypeVar('T') class Copy(Protocol): - def __call__(self, __origin: T) -> T: ... + # '/' marks the end of positional-only parameters + def __call__(self, origin: T, /) -> T: ... copy_a: Callable[[T], T] copy_b: Copy diff --git a/docs/source/runtime_troubles.rst b/docs/source/runtime_troubles.rst index 66ab7b3a84c7..ee4cdf274ebe 100644 --- a/docs/source/runtime_troubles.rst +++ b/docs/source/runtime_troubles.rst @@ -69,7 +69,7 @@ Future annotations import (PEP 563) ----------------------------------- Many of the issues described here are caused by Python trying to evaluate -annotations. Future Python versions (potentially Python 3.12) will by default no +annotations. Future Python versions (potentially Python 3.14) will by default no longer attempt to evaluate function and variable annotations. This behaviour is made available in Python 3.7 and later through the use of ``from __future__ import annotations``. @@ -84,7 +84,7 @@ required to be valid Python syntax. For more details, see :pep:`563`. still require string literals or result in errors, typically involving use of forward references or generics in: - * :ref:`type aliases `; + * :ref:`type aliases ` not defined using the ``type`` statement; * :ref:`type narrowing `; * type definitions (see :py:class:`~typing.TypeVar`, :py:class:`~typing.NewType`, :py:class:`~typing.NamedTuple`); * base classes. @@ -93,6 +93,7 @@ required to be valid Python syntax. For more details, see :pep:`563`. # base class example from __future__ import annotations + class A(tuple['B', 'C']): ... # String literal types needed here class B: ... class C: ... @@ -244,7 +245,8 @@ complicated and you need to use :ref:`typing.TYPE_CHECKING task_queue: Tasks reveal_type(task_queue.get()) # Reveals str -If your subclass is also generic, you can use the following: +If your subclass is also generic, you can use the following (using the +legacy syntax for generic classes): .. code-block:: python @@ -262,9 +264,11 @@ If your subclass is also generic, you can use the following: task_queue: MyQueue[str] reveal_type(task_queue.get()) # Reveals str -In Python 3.9, we can just inherit directly from ``Queue[str]`` or ``Queue[T]`` +In Python 3.9 and later, we can just inherit directly from ``Queue[str]`` or ``Queue[T]`` since its :py:class:`queue.Queue` implements :py:meth:`~object.__class_getitem__`, so -the class object can be subscripted at runtime without issue. +the class object can be subscripted at runtime. You may still encounter issues (even if +you use a recent Python version) when subclassing generic classes defined in third-party +libraries if types are generic only in stubs. Using types defined in stubs but not at runtime ----------------------------------------------- diff --git a/docs/source/type_narrowing.rst b/docs/source/type_narrowing.rst index d698f35c44bc..131171844dfe 100644 --- a/docs/source/type_narrowing.rst +++ b/docs/source/type_narrowing.rst @@ -255,16 +255,13 @@ to the type specified as the first type parameter (``list[str]``). Generic TypeGuards ~~~~~~~~~~~~~~~~~~ -``TypeGuard`` can also work with generic types: +``TypeGuard`` can also work with generic types (Python 3.12 syntax): .. code-block:: python - from typing import TypeVar from typing import TypeGuard # use `typing_extensions` for `python<3.10` - _T = TypeVar("_T") - - def is_two_element_tuple(val: tuple[_T, ...]) -> TypeGuard[tuple[_T, _T]]: + def is_two_element_tuple[T](val: tuple[T, ...]) -> TypeGuard[tuple[T, T]]: return len(val) == 2 def func(names: tuple[str, ...]): @@ -276,16 +273,13 @@ Generic TypeGuards TypeGuards with parameters ~~~~~~~~~~~~~~~~~~~~~~~~~~ -Type guard functions can accept extra arguments: +Type guard functions can accept extra arguments (Python 3.12 syntax): .. code-block:: python - from typing import Type, TypeVar from typing import TypeGuard # use `typing_extensions` for `python<3.10` - _T = TypeVar("_T") - - def is_set_of(val: set[Any], type: Type[_T]) -> TypeGuard[set[_T]]: + def is_set_of[T](val: set[Any], type: type[T]) -> TypeGuard[set[T]]: return all(isinstance(x, type) for x in val) items: set[Any]