Make dict expression inference more consistent

[ilevkivskyi]

Fixes #12977

IMO current dict expression inference logic is quite arbitrary: we only take the non-star items to infer resulting type, then enforce it on the remaining (star) items. In this PR I simplify the logic to simply put all expressions as arguments into the same call. This has following benefits:

• Makes everything more consistent/predictable.
• Fixes one of top upvoted bugs
• Makes dict item indexes correct (previously we reshuffled them causing wrong indexes for non-star items after star items)
• No more weird wordings like List item <n> or Argument <n> to "update" of "dict"
• I also fix the end position of generated expressions to show correct spans in errors

The only downside is that we will see Cannot infer type argument error instead of Incompatible type more often. This is because SupportsKeysAndGetItem (used for star items) is invariant in key type. I think this is fine however, since:

• This only affects key types, that are mixed much less often than value types (they are usually just strings), and for latter we use joins.
• I added a dedicated note for this case

[ilevkivskyi]

Two new errors in black are real (probably we didn't catch it before because dict.update() has some special case for None). Everything else in the primer looks like a clear net positive.

[JelleZijlstra]

Thanks, the Black code is indeed buggy and I submitted a PR to fix it.

Do you know where the None special case is that you mention? I don't see it in https://github.com/python/typeshed/blob/8e24885da7ea20539af099b39336a1197dedc0b8/stdlib/typing.pyi#L633. It seems worth investigating further since it may hide type errors in direct calls to .update() too.

[JelleZijlstra]

Seems like it's not because of the None, it's because the keys were of type Any. This is mypy 1.2.0 behavior:

from typing import Any

_strprefixes: Any
endprogs = {
    "'": "",
    **{prefix: 42 for prefix in _strprefixes},  # no error
}

_notany: set[str]
endprogs2 = {
    "'": "",
    **{prefix: 42 for prefix in _notany},  # E: Value expression in dictionary comprehension has incompatible type "int"; expected type "str"
}

https://mypy-play.net/?mypy=latest&python=3.11&flags=strict&gist=95b51ef3ecb4665f103a5c0c0a2cbe3d

Glad we're fixing this!

[hauntsaninja]

Looks great! Maybe worth adding Jelle's example as a test

[github-actions]

Diff from mypy_primer, showing the effect of this PR on open source code:

sphinx (https://github.com/sphinx-doc/sphinx)
+ sphinx/io.py:185: error: Unused "type: ignore" comment  [unused-ignore]
+ sphinx/io.py:187: error: Unused "type: ignore" comment  [unused-ignore]

prefect (https://github.com/PrefectHQ/prefect)
- src/prefect/infrastructure/base.py:110: error: Argument 1 to "update" of "MutableMapping" has incompatible type "dict[str, str | None]"; expected "SupportsKeysAndGetItem[str, str]"  [arg-type]
- src/prefect/infrastructure/process.py:235: error: Need type annotation for "env" (hint: "env: Dict[<type>, <type>] = ...")  [var-annotated]
- src/prefect/infrastructure/process.py:235: error: List item 0 has incompatible type "object"; expected "SupportsKeysAndGetItem[<nothing>, <nothing>]"  [list-item]
+ src/prefect/infrastructure/process.py:235: error: Unpacked dict entry 0 has incompatible type "object"; expected "SupportsKeysAndGetItem[str, str | None]"  [dict-item]
- src/prefect/infrastructure/process.py:235: error: Argument 1 to "update" of "MutableMapping" has incompatible type "dict[str, str]"; expected "SupportsKeysAndGetItem[<nothing>, <nothing>]"  [arg-type]
- src/prefect/infrastructure/process.py:235: error: Argument 1 to "update" of "MutableMapping" has incompatible type "dict[str, str | None]"; expected "SupportsKeysAndGetItem[<nothing>, <nothing>]"  [arg-type]
- src/prefect/server/models/block_schemas.py:514: error: Need type annotation for "block_schema_fields_copy"  [var-annotated]
- src/prefect/workers/base.py:187: error: Argument 1 to "update" of "MutableMapping" has incompatible type "dict[str, str | None]"; expected "SupportsKeysAndGetItem[str, str]"  [arg-type]

speedrun.com_global_scoreboard_webapp (https://github.com/Avasam/speedrun.com_global_scoreboard_webapp)
- backend/services/user_updater.py:229: error: Argument 2 to "get_file" has incompatible type "dict[str, object]"; expected "SupportsItems[str | bytes | int | float, str | bytes | int | float | Iterable[str | bytes | int | float] | None] | tuple[str | bytes | int | float, str | bytes | int | float | Iterable[str | bytes | int | float] | None] | Iterable[tuple[str | bytes | int | float, str | bytes | int | float | Iterable[str | bytes | int | float] | None]] | str | bytes | None"  [arg-type]

ibis (https://github.com/ibis-project/ibis)
- ibis/backends/bigquery/rewrites.py:27: error: Argument 1 to "update" of "MutableMapping" has incompatible type "dict[Node, Callable[..., Any]]"; expected "SupportsKeysAndGetItem[type[Filterable], Any]"  [arg-type]
+ ibis/backends/bigquery/rewrites.py:26: error: Cannot infer type argument 1 of <dict>  [misc]
+ ibis/backends/bigquery/rewrites.py:26: note: Try assigning the literal to a variable annotated as dict[<key>, <val>]
- ibis/backends/bigquery/compiler.py:70: error: Incompatible types in assignment (expression has type "dict[type[Filterable], Any]", base class "ExprTranslator" defined the type as "dict[Node, Callable[..., Any]]")  [assignment]