Skip to content

Commit d4a83e1

Browse files
committed
Merged master again
2 parents bb5134e + 9aef23b commit d4a83e1

33 files changed

+299
-209
lines changed
Lines changed: 68 additions & 38 deletions
Original file line numberDiff line numberDiff line change
@@ -1,60 +1,90 @@
1-
Function overloading in stubs
2-
=============================
1+
Function Overloading
2+
====================
33

4-
Sometimes you have a library function that seems to call for two or
5-
more signatures. That's okay -- you can define multiple *overloaded*
6-
instances of a function with the same name but different signatures in
7-
a stub file (this feature is not supported for user code, at least not
8-
yet) using the ``@overload`` decorator. For example, we can define an
9-
``abs`` function that works for both ``int`` and ``float`` arguments:
4+
Sometimes the types in a function depend on each other in ways that
5+
can't be captured with a ``Union``. For example, the ``__getitem__``
6+
(``[]`` bracket indexing) method can take an integer and return a
7+
single item, or take a ``slice`` and return a ``Sequence`` of items.
8+
You might be tempted to annotate it like so:
109

1110
.. code-block:: python
1211
13-
# This is a stub file!
14-
15-
from typing import overload
16-
17-
@overload
18-
def abs(n: int) -> int: pass
19-
20-
@overload
21-
def abs(n: float) -> float: pass
22-
23-
Note that we can't use ``Union[int, float]`` as the argument type,
24-
since this wouldn't allow us to express that the return
25-
type depends on the argument type.
26-
27-
Now if we import ``abs`` as defined in the above library stub, we can
28-
write code like this, and the types are inferred correctly:
12+
from typing import Sequence, TypeVar, Union
13+
T = TypeVar('T')
14+
15+
class MyList(Sequence[T]):
16+
def __getitem__(self, index: Union[int, slice]) -> Union[T, Sequence[T]]:
17+
if isinstance(index, int):
18+
... # Return a T here
19+
elif isinstance(index, slice):
20+
... # Return a sequence of Ts here
21+
else:
22+
raise TypeError(...)
23+
24+
But this is too loose, as it implies that when you pass in an ``int``
25+
you might sometimes get out a single item and sometimes a sequence.
26+
The return type depends on the parameter type in a way that can't be
27+
expressed using a type variable. Instead, we can use `overloading
28+
<https://www.python.org/dev/peps/pep-0484/#function-method-overloading>`_
29+
to give the same function multiple type annotations (signatures) and
30+
accurately describe the function's behavior.
2931

3032
.. code-block:: python
3133
32-
n = abs(-2) # 2 (int)
33-
f = abs(-1.5) # 1.5 (float)
34+
from typing import overload, Sequence, TypeVar, Union
35+
T = TypeVar('T')
36+
37+
class MyList(Sequence[T]):
38+
39+
# The @overload definitions are just for the type checker,
40+
# and overwritten by the real implementation below.
41+
@overload
42+
def __getitem__(self, index: int) -> T:
43+
pass # Don't put code here
44+
45+
# All overloads and the implementation must be adjacent
46+
# in the source file, and overload order may matter:
47+
# when two overloads may overlap, the more specific one
48+
# should come first.
49+
@overload
50+
def __getitem__(self, index: slice) -> Sequence[T]:
51+
pass # Don't put code here
52+
53+
# The implementation goes last, without @overload.
54+
# It may or may not have type hints; if it does,
55+
# these are checked against the overload definitions
56+
# as well as against the implementation body.
57+
def __getitem__(self, index):
58+
# This is exactly the same as before.
59+
if isinstance(index, int):
60+
... # Return a T here
61+
elif isinstance(index, slice):
62+
... # Return a sequence of Ts here
63+
else:
64+
raise TypeError(...)
3465
3566
Overloaded function variants are still ordinary Python functions and
36-
they still define a single runtime object. The following code is
37-
thus valid:
38-
39-
.. code-block:: python
40-
41-
my_abs = abs
42-
my_abs(-2) # 2 (int)
43-
my_abs(-1.5) # 1.5 (float)
67+
they still define a single runtime object. There is no automatic
68+
dispatch happening, and you must manually handle the different types
69+
in the implementation (usually with :func:`isinstance` checks, as
70+
shown in the example).
4471

4572
The overload variants must be adjacent in the code. This makes code
4673
clearer, as you don't have to hunt for overload variants across the
4774
file.
4875

76+
Overloads in stub files are exactly the same, except there is no
77+
implementation.
78+
4979
.. note::
5080

5181
As generic type variables are erased at runtime when constructing
5282
instances of generic types, an overloaded function cannot have
5383
variants that only differ in a generic type argument,
54-
e.g. ``List[int]`` versus ``List[str]``.
84+
e.g. ``List[int]`` and ``List[str]``.
5585

5686
.. note::
5787

58-
If you are writing a regular module rather than a stub, you can
59-
often use a type variable with a value restriction to represent
60-
functions as ``abs`` above (see :ref:`type-variable-value-restriction`).
88+
If you just need to constrain a type variable to certain types or
89+
subtypes, you can use a :ref:`value restriction
90+
<type-variable-value-restriction>`.

mypy/checker.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2695,6 +2695,8 @@ def find_isinstance_check(node: Expression,
26952695
return None, {}
26962696
elif isinstance(node, CallExpr):
26972697
if refers_to_fullname(node.callee, 'builtins.isinstance'):
2698+
if len(node.args) != 2: # the error will be reported later
2699+
return {}, {}
26982700
expr = node.args[0]
26992701
if expr.literal == LITERAL_TYPE:
27002702
vartype = type_map[expr]

mypy/checkexpr.py

Lines changed: 20 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -151,7 +151,7 @@ def analyze_ref_expr(self, e: RefExpr, lvalue: bool = False) -> Type:
151151
result = type_object_type(node, self.named_type)
152152
elif isinstance(node, MypyFile):
153153
# Reference to a module object.
154-
result = self.named_type('builtins.module')
154+
result = self.named_type('types.ModuleType')
155155
elif isinstance(node, Decorator):
156156
result = self.analyze_var_ref(node.var, e)
157157
else:
@@ -178,6 +178,19 @@ def visit_call_expr(self, e: CallExpr, allow_none_return: bool = False) -> Type:
178178
e.callee.node.typeddict_type is not None:
179179
return self.check_typeddict_call(e.callee.node.typeddict_type,
180180
e.arg_kinds, e.arg_names, e.args, e)
181+
if isinstance(e.callee, NameExpr) and e.callee.name in ('isinstance', 'issubclass'):
182+
for typ in mypy.checker.flatten(e.args[1]):
183+
if isinstance(typ, NameExpr):
184+
try:
185+
node = self.chk.lookup_qualified(typ.name)
186+
except KeyError:
187+
# Undefined names should already be reported in semantic analysis.
188+
node = None
189+
if (isinstance(typ, IndexExpr)
190+
and isinstance(typ.analyzed, (TypeApplication, TypeAliasExpr))
191+
# node.kind == TYPE_ALIAS only for aliases like It = Iterable[int].
192+
or isinstance(typ, NameExpr) and node and node.kind == nodes.TYPE_ALIAS):
193+
self.msg.type_arguments_not_allowed(e)
181194
self.try_infer_partial_type(e)
182195
callee_type = self.accept(e.callee)
183196
if (self.chk.options.disallow_untyped_calls and
@@ -2375,24 +2388,24 @@ def replace_callable_return_type(c: CallableType, new_ret_type: Type) -> Callabl
23752388
return c.copy_modified(ret_type=new_ret_type)
23762389

23772390

2378-
class ArgInferSecondPassQuery(types.TypeQuery):
2391+
class ArgInferSecondPassQuery(types.TypeQuery[bool]):
23792392
"""Query whether an argument type should be inferred in the second pass.
23802393
23812394
The result is True if the type has a type variable in a callable return
23822395
type anywhere. For example, the result for Callable[[], T] is True if t is
23832396
a type variable.
23842397
"""
23852398
def __init__(self) -> None:
2386-
super().__init__(False, types.ANY_TYPE_STRATEGY)
2399+
super().__init__(any)
23872400

23882401
def visit_callable_type(self, t: CallableType) -> bool:
23892402
return self.query_types(t.arg_types) or t.accept(HasTypeVarQuery())
23902403

23912404

2392-
class HasTypeVarQuery(types.TypeQuery):
2405+
class HasTypeVarQuery(types.TypeQuery[bool]):
23932406
"""Visitor for querying whether a type has a type variable component."""
23942407
def __init__(self) -> None:
2395-
super().__init__(False, types.ANY_TYPE_STRATEGY)
2408+
super().__init__(any)
23962409

23972410
def visit_type_var(self, t: TypeVarType) -> bool:
23982411
return True
@@ -2402,10 +2415,10 @@ def has_erased_component(t: Type) -> bool:
24022415
return t is not None and t.accept(HasErasedComponentsQuery())
24032416

24042417

2405-
class HasErasedComponentsQuery(types.TypeQuery):
2418+
class HasErasedComponentsQuery(types.TypeQuery[bool]):
24062419
"""Visitor for querying whether a type has an erased component."""
24072420
def __init__(self) -> None:
2408-
super().__init__(False, types.ANY_TYPE_STRATEGY)
2421+
super().__init__(any)
24092422

24102423
def visit_erased_type(self, t: ErasedType) -> bool:
24112424
return True

mypy/checkmember.py

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -415,7 +415,7 @@ def analyze_class_attribute_access(itype: Instance,
415415

416416
if isinstance(node.node, MypyFile):
417417
# Reference to a module object.
418-
return builtin_type('builtins.module')
418+
return builtin_type('types.ModuleType')
419419

420420
if is_decorated:
421421
# TODO: Return type of decorated function. This is quick hack to work around #998.
@@ -632,11 +632,14 @@ def expand(target: Type) -> Type:
632632
arg_types = func.arg_types[1:]
633633
ret_type = func.ret_type
634634
variables = func.variables
635+
if isinstance(original_type, CallableType) and original_type.is_type_obj():
636+
original_type = TypeType(original_type.ret_type)
635637
res = func.copy_modified(arg_types=arg_types,
636638
arg_kinds=func.arg_kinds[1:],
637639
arg_names=func.arg_names[1:],
638640
variables=variables,
639-
ret_type=ret_type)
641+
ret_type=ret_type,
642+
bound_args=[original_type])
640643
return cast(F, res)
641644

642645

mypy/constraints.py

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -6,8 +6,7 @@
66
from mypy.types import (
77
CallableType, Type, TypeVisitor, UnboundType, AnyType, NoneTyp, TypeVarType,
88
Instance, TupleType, TypedDictType, UnionType, Overloaded, ErasedType, PartialType,
9-
DeletedType, UninhabitedType, TypeType, TypeVarId, TypeQuery, ALL_TYPES_STRATEGY,
10-
is_named_instance
9+
DeletedType, UninhabitedType, TypeType, TypeVarId, TypeQuery, is_named_instance
1110
)
1211
from mypy.maptype import map_instance_to_supertype
1312
from mypy import nodes
@@ -250,9 +249,9 @@ def is_complete_type(typ: Type) -> bool:
250249
return typ.accept(CompleteTypeVisitor())
251250

252251

253-
class CompleteTypeVisitor(TypeQuery):
252+
class CompleteTypeVisitor(TypeQuery[bool]):
254253
def __init__(self) -> None:
255-
super().__init__(default=True, strategy=ALL_TYPES_STRATEGY)
254+
super().__init__(all)
256255

257256
def visit_none_type(self, t: NoneTyp) -> bool:
258257
return experiments.STRICT_OPTIONAL

mypy/messages.py

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -485,7 +485,10 @@ def incompatible_argument(self, n: int, m: int, callee: CallableType, arg_type:
485485
target = ''
486486
if callee.name:
487487
name = callee.name
488-
base = extract_type(name)
488+
if callee.bound_args and callee.bound_args[0] is not None:
489+
base = self.format(callee.bound_args[0])
490+
else:
491+
base = extract_type(name)
489492

490493
for op, method in op_methods.items():
491494
for variant in method, '__r' + method[2:]:
@@ -878,6 +881,9 @@ def typeddict_item_name_not_found(self,
878881
self.fail('\'{}\' is not a valid item name; expected one of {}'.format(
879882
item_name, format_item_name_list(typ.items.keys())), context)
880883

884+
def type_arguments_not_allowed(self, context: Context) -> None:
885+
self.fail('Parameterized generics cannot be used with class or instance checks', context)
886+
881887

882888
def capitalize(s: str) -> str:
883889
"""Capitalize the first character of a string."""

mypy/nodes.py

Lines changed: 3 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@
99

1010
import mypy.strconv
1111
from mypy.visitor import NodeVisitor, StatementVisitor, ExpressionVisitor
12-
from mypy.util import short_type, IdMapper
12+
from mypy.util import short_type
1313

1414

1515
class Context:
@@ -612,7 +612,6 @@ class Decorator(SymbolNode, Statement):
612612
func = None # type: FuncDef # Decorated function
613613
decorators = None # type: List[Expression] # Decorators, at least one # XXX Not true
614614
var = None # type: Var # Represents the decorated function obj
615-
type = None # type: mypy.types.Type
616615
is_overload = False
617616

618617
def __init__(self, func: FuncDef, decorators: List[Expression],
@@ -1988,9 +1987,6 @@ class is generic then it will be a type constructor of higher kind.
19881987
# Is this a newtype type?
19891988
is_newtype = False
19901989

1991-
# Alternative to fullname() for 'anonymous' classes.
1992-
alt_fullname = None # type: Optional[str]
1993-
19941990
FLAGS = [
19951991
'is_abstract', 'is_enum', 'fallback_to_any', 'is_named_tuple',
19961992
'is_newtype'
@@ -2171,8 +2167,7 @@ def serialize(self) -> JsonDict:
21712167
data = {'.class': 'TypeInfo',
21722168
'module_name': self.module_name,
21732169
'fullname': self.fullname(),
2174-
'alt_fullname': self.alt_fullname,
2175-
'names': self.names.serialize(self.alt_fullname or self.fullname()),
2170+
'names': self.names.serialize(self.fullname()),
21762171
'defn': self.defn.serialize(),
21772172
'abstract_attributes': self.abstract_attributes,
21782173
'type_vars': self.type_vars,
@@ -2194,7 +2189,6 @@ def deserialize(cls, data: JsonDict) -> 'TypeInfo':
21942189
module_name = data['module_name']
21952190
ti = TypeInfo(names, defn, module_name)
21962191
ti._fullname = data['fullname']
2197-
ti.alt_fullname = data['alt_fullname']
21982192
# TODO: Is there a reason to reconstruct ti.subtypes?
21992193
ti.abstract_attributes = data['abstract_attributes']
22002194
ti.type_vars = data['type_vars']
@@ -2303,14 +2297,7 @@ def serialize(self, prefix: str, name: str) -> JsonDict:
23032297
else:
23042298
if self.node is not None:
23052299
if prefix is not None:
2306-
# Check whether this is an alias for another object.
2307-
# If the object's canonical full name differs from
2308-
# the full name computed from prefix and name,
2309-
# it's an alias, and we serialize it as a cross ref.
2310-
if isinstance(self.node, TypeInfo):
2311-
fullname = self.node.alt_fullname or self.node.fullname()
2312-
else:
2313-
fullname = self.node.fullname()
2300+
fullname = self.node.fullname()
23142301
if (fullname is not None and '.' in fullname and
23152302
fullname != prefix + '.' + name):
23162303
data['cross_ref'] = fullname

mypy/semanal.py

Lines changed: 11 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -113,6 +113,13 @@
113113
'typing.typevar': 'typing.TypeVar',
114114
}
115115

116+
# Rename objects placed in _importlib_modulespec due to circular imports
117+
module_rename_map = {
118+
'_importlib_modulespec.ModuleType': 'types.ModuleType',
119+
'_importlib_modulespec.ModuleSpec': 'importlib.machinery.ModuleSpec',
120+
'_importlib_modulespec.Loader': 'importlib.abc.Loader'
121+
}
122+
116123
# Hard coded type promotions (shared between all Python versions).
117124
# These add extra ad-hoc edges to the subtyping relation. For example,
118125
# int is considered a subtype of float, even though there is no
@@ -258,8 +265,8 @@ def visit_file(self, file_node: MypyFile, fnam: str, options: Options) -> None:
258265

259266
if self.cur_mod_id == 'builtins':
260267
remove_imported_names_from_symtable(self.globals, 'builtins')
261-
for alias_name in ['List', 'Dict', 'Set']:
262-
self.globals.pop(alias_name, None)
268+
for alias_name in type_aliases:
269+
self.globals.pop(alias_name.split('.')[-1], None)
263270

264271
if '__all__' in self.globals:
265272
for name, g in self.globals.items():
@@ -2563,7 +2570,6 @@ def visit_decorator(self, dec: Decorator) -> None:
25632570
self.fail('Too many arguments', dec.func)
25642571
elif refers_to_fullname(d, 'typing.no_type_check'):
25652572
dec.var.type = AnyType()
2566-
dec.type = dec.var.type
25672573
no_type_check = True
25682574
for i in reversed(removed):
25692575
del dec.decorators[i]
@@ -3446,6 +3452,8 @@ def visit_file(self, file: MypyFile, fnam: str, mod_id: str, options: Options) -
34463452

34473453
for d in defs:
34483454
d.accept(self)
3455+
if isinstance(d, ClassDef):
3456+
d.info._fullname = module_rename_map.get(d.info._fullname, d.info._fullname)
34493457

34503458
# Add implicit definition of literals/keywords to builtins, as we
34513459
# cannot define a variable with them explicitly.

mypy/server/astdiff.py

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -81,7 +81,6 @@ def is_similar_node_shallow(n: SymbolTableNode, m: SymbolTableNode) -> bool:
8181
nn.fallback_to_any == mn.fallback_to_any and
8282
nn.is_named_tuple == mn.is_named_tuple and
8383
nn.is_newtype == mn.is_newtype and
84-
nn.alt_fullname == mn.alt_fullname and
8584
is_same_mro(nn.mro, mn.mro))
8685
if isinstance(n.node, Var) and isinstance(m.node, Var):
8786
return is_identical_type(n.node.type, m.node.type)

0 commit comments

Comments
 (0)