Crash using NamedTuple containing generic Callable

[PeterJCLaw]

Crash Report

Given a source file containing:

class ProgressLogger(NamedTuple):
    progress: Callable[[T], T]

where that type is used in another file as an annotation:

def bar(progress_logger: ProgressLogger) -> None:
    pass

Note that this doesn't reproduce if the function is within the same file as the type definitions. In that case mypy completes without complaint.

Traceback

# Using mypy from git at 92923b2ed8085a38c353382f805c9e2e8716e717
(mypy) peter@Tarasque:mypy (master)$ mypy /tmp/repro/project/ --show-traceback
/tmp/repro/project/consumer.py:4: error: INTERNAL ERROR -- Please try using mypy master on Github:
https://mypy.rtfd.io/en/latest/common_issues.html#using-a-development-mypy-build
Please report a bug at https://github.com/python/mypy/issues
version: 0.800+dev.92923b2ed8085a38c353382f805c9e2e8716e717
Traceback (most recent call last):
  File "/home/peter/.virtualenvs/mypy/bin/mypy", line 33, in <module>
    sys.exit(load_entry_point('mypy', 'console_scripts', 'mypy')())
  File "/home/play/mypy/mypy/__main__.py", line 11, in console_entry
    main(None, sys.stdout, sys.stderr)
  File "/home/play/mypy/mypy/main.py", line 90, in main
    res = build.build(sources, options, None, flush_errors, fscache, stdout, stderr)
  File "/home/play/mypy/mypy/build.py", line 180, in build
    result = _build(
  File "/home/play/mypy/mypy/build.py", line 254, in _build
    graph = dispatch(sources, manager, stdout)
  File "/home/play/mypy/mypy/build.py", line 2638, in dispatch
    process_graph(graph, manager)
  File "/home/play/mypy/mypy/build.py", line 2964, in process_graph
    process_stale_scc(graph, scc, manager)
  File "/home/play/mypy/mypy/build.py", line 3056, in process_stale_scc
    mypy.semanal_main.semantic_analysis_for_scc(graph, scc, manager.errors)
  File "/home/play/mypy/mypy/semanal_main.py", line 79, in semantic_analysis_for_scc
    process_functions(graph, scc, patches)
  File "/home/play/mypy/mypy/semanal_main.py", line 233, in process_functions
    process_top_level_function(analyzer,
  File "/home/play/mypy/mypy/semanal_main.py", line 274, in process_top_level_function
    deferred, incomplete, progress = semantic_analyze_target(target, state, node, active_type,
  File "/home/play/mypy/mypy/semanal_main.py", line 326, in semantic_analyze_target
    analyzer.refresh_partial(refresh_node,
  File "/home/play/mypy/mypy/semanal.py", line 397, in refresh_partial
    self.accept(node)
  File "/home/play/mypy/mypy/semanal.py", line 4835, in accept
    node.accept(self)
  File "/home/play/mypy/mypy/nodes.py", line 687, in accept
    return visitor.visit_func_def(self)
  File "/home/play/mypy/mypy/semanal.py", line 573, in visit_func_def
    self.analyze_func_def(defn)
  File "/home/play/mypy/mypy/semanal.py", line 602, in analyze_func_def
    result = analyzer.visit_callable_type(defn.type, nested=False)
  File "/home/play/mypy/mypy/typeanal.py", line 527, in visit_callable_type
    ret = t.copy_modified(arg_types=self.anal_array(t.arg_types, nested=nested),
  File "/home/play/mypy/mypy/typeanal.py", line 929, in anal_array
    res.append(self.anal_type(t, nested))
  File "/home/play/mypy/mypy/typeanal.py", line 936, in anal_type
    return t.accept(self)
  File "/home/play/mypy/mypy/types.py", line 490, in accept
    return visitor.visit_unbound_type(self)
  File "/home/play/mypy/mypy/typeanal.py", line 163, in visit_unbound_type
    typ = self.visit_unbound_type_nonoptional(t, defining_literal)
  File "/home/play/mypy/mypy/typeanal.py", line 252, in visit_unbound_type_nonoptional
    return self.analyze_type_with_type_info(node, t.args, t)
  File "/home/play/mypy/mypy/typeanal.py", line 385, in analyze_type_with_type_info
    return tup.copy_modified(items=self.anal_array(tup.items),
  File "/home/play/mypy/mypy/typeanal.py", line 929, in anal_array
    res.append(self.anal_type(t, nested))
  File "/home/play/mypy/mypy/typeanal.py", line 936, in anal_type
    return t.accept(self)
  File "/home/play/mypy/mypy/types.py", line 1137, in accept
    return visitor.visit_callable_type(self)
  File "/home/play/mypy/mypy/typeanal.py", line 526, in visit_callable_type
    variables = self.bind_function_type_variables(t, t)
  File "/home/play/mypy/mypy/typeanal.py", line 900, in bind_function_type_variables
    assert var_node, "Binding for function type variable not found within function"
AssertionError: Binding for function type variable not found within function
/tmp/repro/project/consumer.py:4: : note: use --pdb to drop into pdb

To Reproduce

Initially observed in thread/django-lightweight-queue#42, specifically https://github.com/thread/django-lightweight-queue/blob/7c8a2dc066acdf22bdfad88f2d2519f90155f209/django_lightweight_queue/backends/reliable_redis.py#L114-L119.

Cut-down to: https://gist.github.com/PeterJCLaw/00b0f5c5a5a6abcf53d2b6d6a8d142c6

Your Environment

Initially observed under Python 3.5.10, Mypy 0.770. Reproduced under Mypy 0.790.
Also reproduced on master at 92923b2 under Python 3.8.5.

OS: Ubuntu 20.04.1.

[PeterJCLaw]

I've just found a workaround to this -- by adding a TypeVar declaration at the top of the file which uses the NamedTuple where the name of the TypeVar matches the name of the one used in the Callable, mypy is then happy with the source.

[Michael0x2a]

Looks like this was fixed by #13371, according to git-bisect.

I'm not exactly sure how that diff addressed this problem, but iiuc it basically involved making mypy handle namedtuples a little more lazily to help support recursive types. I think it's plausible that change was sufficient to fix this crash.