Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

stubgen: fixes and simplifications #15232

Merged
merged 2 commits into from
May 18, 2023
Merged

Conversation

hamdanal
Copy link
Collaborator

This PR refactors name resolution in stubgen to make it simple to use and fixes a bunch of related bugs especially when run in --parse-only mode. It mainly does the following:

  1. Adds a get_fullname method that resolves the full name of both NameExpr and MemberExpr taking into account import aliases
  2. Use this method everywhere manual resolution was done including the decorator processors
  3. Simplify decorator processing by consolidating process_decorator, process_name_expr_decorator, and process_member_expr_decorator into a single method thanks to the common name resolution using get_fullname
  4. Fix occurrences of some hard-coded implicitly-added names (like Incomplete from _typeshed) by using self.typing_name() and self.add_typing_import which take into account objects of the same name defined in the file
  5. Fix some inconsistencies in generated empty lines. This improves the consistency within stubgen as well as with the black code style for stubs.

Out of the 9 test cases added, only testAbstractPropertyImportAlias passes on master (added it any way because it was not tested). The other 8 all fail without this PR. The last test case testUseTypingName demonstrates point 4.. In existing test cases, only white space changes in four of the cases due to point 5. above were made.

This will allow easier additions in the future as no manual name resolution has to be made any more.

@JelleZijlstra
Copy link
Member

There's a CI failure (apparently only when mypyc is enabled).

@hamdanal
Copy link
Collaborator Author

This turned out to be a python 3.11 failure because asyncio.coroutine was removed in 3.11 and mypy couldn't resolve the import. get_fullname now falls back to non-semantic analysis resolution when it detects a suppressed import because stubgen should still generate the stub correctly even when its run with a newer version of python

Copy link
Collaborator

@hauntsaninja hauntsaninja left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, this cleans up several things. I'll leave open for a couple days in case someone wants to take a second look (many lines touched, so it's easy to miss things in review).

@JelleZijlstra JelleZijlstra merged commit d98cc5a into python:master May 18, 2023
@hamdanal hamdanal deleted the stubgen-fix branch May 19, 2023 18:05
hamdanal added a commit to hamdanal/mypy that referenced this pull request Oct 21, 2023
The semantic analyzer treats properties as overloaded functions.
This was previously ignored by stubgen but regressed in python#15232.

Fixes python#16300
JelleZijlstra pushed a commit that referenced this pull request Oct 21, 2023
The semantic analyzer treats properties as overloaded functions. This
was previously ignored by stubgen but regressed in #15232.
This PR restores the original behavior.

Fixes #16300
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants