Python existed for over 20 years without type annotations, so why now do most new libraries come fully annotated? What changed?
Python is a dynamically typed language. In practice this means that what we think of as variables holding data are actually names referring to data.
When we see code like:
status_code = "300" # presumably read from file/network
status_code = int(status_code) # did the variable change types?In fact, there are two different objects here:
status_code = "300" # presumably read from file/network
print(id(status_code), type(status_code))
status_code = int(status_code) # did the variable change types?
print(id(status_code), type(status_code))140228278133472 <class 'str'>
140228279459984 <class 'int'>id shows us these refer to two different memory locations. When we do assignment, the name moves to point at the new location.
This is fundamentally different from how variables & names work in many languages– it makes Python a very flexible language, but can lead to harder to understand code that poses maintenance challenges.
Type annotations were introduced in Python 3.5 PEP 484, which notably contains the line:
It should also be emphasized that Python will remain a dynamically typed language, and the authors have no desire to ever make type hints mandatory, even by convention.
(emphasis retained from original)
This aims to head-off the most common misconception about types in Python, that they are becoming mandatory. In fact– most Python developers recognize, they are not needed for all code, and deciding if they are suited for your use case takes understanding why they were added in the first place.
The second misconception we need to dispel is that type annotations are in any way enforced. Type annotations have all the power of a docstring, that is to say, not much at all.
def bad_annotation(x: int, y: str) -> None:
# this function treats x like a list/sequence, y like an integer,
# and returns a value -- contrary to its annotation
return x[y - 1]
bad_annotation([1, 2, 3], 0)3This code works– no exceptions or errors will be raised. The annotations are just wrong. No runtime checking is performed on type annotations.
So why have them in the first place?
The key idea here is that they can be accessed, let’s take a look:
bad_annotation.__annotations__{'x': int, 'y': str, 'return': None}Having access to this information in a structured manner gives us options! We could write a decorator function that enforces the types, external tools that validate our code, and much more.
Before we dive deeper into all of the things we can use type annotations for, let’s take a look at how to write them.
We can write annotations in three places:
def find_case_insensitive(
search: str,
items: list[str],
) -> int:
""" returns count of how many times search appears in items """Each parameter can have an annotation, with special -> syntax for annotating return types.
These can be accessed via func.__annotations__:
find_case_insensitive.__annotations__{'search': str, 'items': list[str], 'return': int}Class annotations exist on the body of the class, they do not create variables, but provide annotations for attributes typically set in __init__.
class Vector2:
# annotates that these class attributes are float
x: float
y: float
# self does not need an annotation
# __init__ is always meant to be annotated as returning None
def __init__(self, x: float, y: float) -> None:
self.x = x
self.y = yThese can be accessed as Vector2.__annotations__:
Vector2.__annotations__{'x': float, 'y': float}It is also possible to annotate variables at any scope:
APPROX_PI: float = 22/7
def func():
guesses: list[float] = []
...These annotations are accessed through a scope-level __annotations__, and are typically present to satisfy type checkers, whereas function & class-level annotations are often used programmatically as well.
An annotation can be any valid Python expression. While most tools will expect that expression to evaluate to a type, there is no such requirement:
# unusual annotations
def f(x : 3, y: "test".upper(), z: min(1, 2, 3)) -> False:
pass
f.__annotations__{'x': 3, 'y': 'TEST', 'z': 1, 'return': False}Of course, a type checker will complain about these non-type annotations, but knowing we can use expressions can give us flexibility. An annotation can be the return value of a function, or another variable.
Despite accepting anything as an annotation there are a set of types an expressions that are meant to be used to annotate functions & classes.
These have evolved rapidly between Python 3.5 and now, many of the enhancements between 3.6 and now have added ways to more easily type common cases and/or to more specifically capture patterns at the edge of Python’s dynamic nature.
It is common for a function argument to take multiple types, the | union operator can be used to easily express this idea:
def takes_numeric(x: int | float, y: int | float) -> int | float:
passIn older versions of Python this was written typing.Union[int, float].
Relatedly, if a parameter can be int or None it can be annotated as int | None, this replaces typing.Optional[int].
In new code, prefer the new syntax, but you will see Union and Optional in older code and documentation.
Container types can include details on what they contain:
list[int] - List of integers.list[int | float] - List of ints & floats.dict[str, int] - Dictionary mapping string keys to int values.tuple[int, str, str] - Tuple with three elements: one int, two str.These, as type values themselves, can be arbitrarily nested:
dict[str, list[int]] - Dictionary mapping string keys to lists of ints.dict[tuple[float, float], list[dict[str, str]]] - dictionary mapping (lat, lon) pairs to list of airports represented as str -> str mappings.There was an intermediate syntax for this as well, typing.List[int], typing.Dict[str, str], etc. These capitalized-forms should no longer be used for built-in types. Use the type name like list, dict, set directly.
As seen above, these can quickly get pretty complex. We can create type aliases using the type keyword:
type OptionDict = dict[str, str | int]
type Vector3 = tuple[float, float, float]
type RequestPayload = tuple[Vector3, OptionDict, str]
def send_request(req: RequestPayload) -> OptionDict:
...This can make our code more readable, and give users reading the code a better understanding of our intent than tuple[tuple[float, float, float], dict[str, str|int]] ever would.
Lots of people appreciate type annotations until it comes time to annotate something beyond the core & compound types. Python’s dynamic nature makes it possible to write very expressive code, but the types of that code can be harder to reason about.
Generator functions can be annotated with Generator[YieldType, SendType=None, ReturnType=None]:
import typing
def powers_of_two() -> Generator[int]:
n = 0
while True:
yield 2**n
n += 1The other two parameters are used for full coroutines, where values are being sent back & forth:
def echo_round() -> Generator[int, float, str]:
sent = yield 0
while sent >= 0:
sent = yield round(sent)
return 'Done'(We’ll discuss coroutines later in this course.)
https://typing.python.org/en/latest/spec/typeddict.html
A dict is typically typed as dict[key_type, val_type], but this leaves quite a bit to be desired for the common case where a given key has a specific type.
TypedDict allows defining dictionaries:
from typing import TypedDict
class UserData(TypedDict):
id: int
username: str
email: str
def get_user_list(...) -> list[UserData]:
"""
Example Response:
"""
return [
{"id": 1, "username": "admin", "email": "admin@example.com"},
{"id": 2, "username": "root", "email": "root@example.com"},
]
...When writing functional code it is common to pass functions to other functions, we annotate these with collections.abc.Callable (formerly typing.Callable )
from collections.abc import Callable
def modify_list_in_place(strings: list[str], f: Callable[[str], str]):
for i, item in enumerate(strings):
# f is a function that is str -> str
strings[i] = f(item)The syntax for Callable is a bit different, it takes two generic parameters, the first of which is a list of types representing the parameters (here a single str), and the second being the return type.
def split(self, sep: str | None, maxsplit: int) -> list[str]from typing import Any, Callable
def sorted(iterable: list[Any],
key: Callable[[Any], Any] | None = None,
reverse: bool = False) -> list[Any]:
# *close, but we'll see a better solution to list belowT = TypeVar("T")
def enumerate(iter: Iterable[T]) -> Iterator[tuple[int, T]]from typing import TypeVar, overload
T = TypeVar("T")
R = TypeVar("R")
def map(func: Callable[[T1], R], iter1: Iterable[T1]) -> Iterator[R]: ...
@overload
def map(func: Callable[[T1, T2], R], iter1: Iterable[T1], iter2: Iterable[T2]) -> Iterator[R]: ...
@overload
def map(func: Callable[[T1, T2, T3], R], iter1: Iterable[T1], iter2: Iterable[T2], iter3: Iterable[T3]) -> Iterator[R]: ...
# again, approximate, actual method is even more complex
# due to class implementation under the hoodIn the example with sorted we assumed input was a list but in practice we know sorted and functions like it can take any iterable.
collections.abc provides many of these protocol-based abstract types:
ContainerHashableIterableSizedCallableCollectionSequenceMappingThee can be used in lieu of the more specific types when appropriate, so we would instead write sorted’s annotation as:
def sorted(iterable: Iterable[Any],
key: Callable[[Any], Any] | None = None,
reverse: bool = False) -> list[Any]:TypeVar genericsBut this isn’t as specific as we may want either. The actual types in the iterable may be Any, but this would allow a list like [1, "two", 3.0, None] – we may want to say all elements are of the same type, and that we know the key callable takes that type.
from typing import TypeVar
# creates a new "generic" type -- this type is not yet known
# but we are saying all occurrences of T will be the same type
T = TypeVar("T")
def sorted(iterable: Iterable[T],
key: Callable[[T], T] | None = None,
reverse: bool = False) -> list[T]:We realize that our callable does not need to return the same type as its input (e.g. we are sorting tuples by a string key), we would introduce a second generic type:
from typing import TypeVar
# creates a new "generic" type -- this type is not yet known
# but we are saying all occurrences of T will be the same type
T = TypeVar("T")
S = TypeVar("S")
def sorted(iterable: Iterable[T],
key: Callable[[T], S] | None = None,
reverse: bool = False) -> list[T]:Never indicates a function that should not return without an exception:
def game_loop() -> Never:
""" only exits when StopGame is raised from within update_state """
while True:
update_state()
draw_screen()Literal acts as a type-check only Enum, useful for documenting older APIs:
TempUnit = Literal["C", "F"]
def show_temp(unit: TempUnit):
...(An Enum/StrEnum is almost always a better choice!)
Final indicates a value will not change:
MAX_ROWS: Final = 50
MAX_ROWS = 10 # will be flagged by type checkerIn most object-oriented languages we describe the relationship between a class and a derived/child class as an “is-a” relationship.
# caching plugin is-a plugin
class CachingPlugin(Plugin):
...
# anywhere a plugin is expected, CachingPlugin is welcome
def register_plugin(p: Plugin) -> bool:
...This is called nominal subtyping, CachingPlugin is a Plugin because we said so.
Often in Python, we use duck typing, also known as structural subtyping, where a type is substitutable for another because it has the same interface or protocol.
We can express this in the type system as well:
from typing import Protocol
# this is the complete protocol definition, we define "stubs"
# that any Plugin needs to implement
class Plugin(Protocol):
def augment_request(self, r: Request) -> Request: ...
def format_report(self) -> str: ...
class CachingPlugin:
def augment_request(self, r: Request) -> Request:
if resp := self._cache.get(r.key):
return resp
else:
...
def format_report(self) -> str:
return "..."No direct inheritance relationship, but Python and type checkers will recognize CachingPlugin as fulfilling the plugin Protocol.
For more details on advanced type annotations see the linked additional resources which cover even more advanced cases necessary when writing particularly dynamic code.
The most conventional use of type annotations is to augment the existing linting of one’s code. This is particularly important for library code, or code in large projects where many developers will interact with it– and can’t be expected to fully understand what to pass to large numbers of untyped functions.
Python type checkers support gradual typing, typically only pointing out issues in code that has been annotated. If a parameter or attribute does not have annotations it is typically assumed to take typing.Any.
There are three type checkers in common use:
mypyWritten in Python, most complete, highly configurable.
A good option for running simple checks from the command line, considered the reference implementation as it is written by the Python core team.
Typically run via mypy file.py or mypy module/
pyrightSlightly faster, maintained by Microsoft. Typically enabled within VS Code to enable in-editor type checking via LSP (Language Server Protocol, common interface for editor/linter communication.)
A good fit if you prefer editor integration. Can be run on CLI like mypy.
tyPart of the Astral toolchain like uv and ruff. Written in Rust, faster by 10x or more, important for very large codebases.
Intentionally less configurable, is doing formal type checking, not mere linting. Allows for gradual typing of untyped code.
Still in beta, likely to become as ubiquitous as uv/ruff. (Possibly even integrated into ruff in a couple of years.)
Type checkers do more than just look at the raw types, utilizing some of the logic in the program to understand what type(s) might be present on a given branch of execution.
# this code has an error can you see it?
def some_func(items: list[str], compare: str | None = None) -> bool:
for item in items:
if item == compare.lower():
return True
return False# fixed, and type checker is satisfied too
def some_func(items: list[str], compare: str | None = None) -> bool:
if compare is None:
compare = "default-string"
else:
# not a type error because this branch only runs if compare is str
compare = compare.lower()
for item in items:
if item == compare:
return True
return FalseType-narrowing in combination with a type checker can help catch many runtime bugs before they occur.
It is possible to define functions that narrow types using typing.TypeGuard:
from typing import TypeGuard
def all_resolved(val: list[int|User]) -> TypeGuard[list[User]]:
return all(isinstance(x, User) for x in val)
# might return ids or cached users
some_list: list[int | User] = get_users()
if all_resolved(some_list) and len(some_list):
some_list[0].username # can safely access User attributesSometimes it is necessary to disable the type checker: either temporarily while you work to refine an interface, or more permanently if a particularly complex piece of code just doesn’t agree with what can be checked.
from typing import cast
# typing.cast is a lie to the type checker, not a true type cast
# x will store an int, but the type checker will not complain
x: str = cast(str, 42)
str.upper(x) # type checker will not complain, but still invalid @ runtime!
# meant to be used to narrow known types
resp = some_api_resp() # returns dict[str, object]
name = cast(str, resp["name"]) # asserting "I know this is a string"# type: ignore disables type checking on a line:
from some_old_lib import func
# the library is incorrectly typed, so we just disable checking here
x: int = func("A") # type: ignorePython programmers have been finding many clever & valuable uses of annotations beyond type checking. As we saw, __annotations__ gives us run time access to additional information we can leverage:
from pydantic import BaseModel
class User(BaseModel):
name: str
age: int
# unlike dataclasses, Pydantic's BaseModel uses `__annotations__` to validate type
try:
user = User(name="Alice", age="fifty")
except Exception as e:
print(e)1 validation error for User
age
Input should be a valid integer, unable to parse string as an integer [type=int_parsing, input_value='fifty', input_type=str]
For further information visit https://errors.pydantic.dev/2.10/v/int_parsingTyper uses annotations to define command line arguments:
# repeat.py
cli = typer.Typer()
@cli.command()
def repeat(message: str, count: int = 1):
for _ in range(count):
print(f"{message}")$ python repeat.py hello --count 3
hello
hello
helloAnd FastAPI does this for HTTP APIs:
app = FastAPI()
@app.get("/users/{user_id}")
def get_user(user_id: int, active: bool = True) -> User:
...
# accessed via
# /users/1
# /users/1?active=falseOfficial Python Typing Docs: https://docs.python.org/3/library/typing.html
Full Specification: https://typing.python.org/en/latest/
MyPy Cheat Sheet: https://mypy.readthedocs.io/en/stable/cheat_sheet_py3.html