Subtyping and TypedDict

[decorator-factory]

from typing import Union, TypedDict

class Foo(TypedDict):
    x: int

class Foo2(Foo):
    y: int

class Bar(TypedDict):
    y: str

def f(foobar: Union[Foo, Bar]):
    if 'y' in foobar:
        print(foobar['y'].lower())

x: Foo2 = {'x': 1, 'y': 2}
f(x)

This will fail at runtime, but typechecks just fine. That's because Foo is not guaranteed to not have a y key, as it can be a subtype of Foo. Is this as designed?

[erictraut]

That's an interesting case.

Pyright's type narrowing logic for the <literal string> in <TypedDict> pattern is eliminating Foo from the union. This type narrowing pattern becomes much less useful if we need to make the assumption that any dictionary key that is absent in a TypedDict might be added in a subclass. I think our options are 1) live with this small type checking hole, or 2) apply the type narrowing only for @final TypedDict classes or 3) completely eliminate this type narrowing pattern and live with the false positive errors that may result. When forced to choose between false negatives and false positives, we generally choose to eliminate the false negative.

It appears that mypy does not provide type narrowing for this pattern, and maybe this is why.

I'm interested in any thoughts from @JukkaL or @gvanrossum, the author and sponsor of PEP 589.

[decorator-factory]

For context, I came up with this example when someone else showed me that this type-checks in TypeScript:

type A =
  | { a: true, f: () => void }
  | { b: true }

function f(x: A) {
    if ("a" in x) {
        x.f()
    }
}

const x: A = { a: true, b: true }

f(x)

[gvanrossum]

I don't know what to say. I think mypy is right to flag this as an error. I also think that this pattern is much less important in Python than it is in TS -- you should probably be using real classes and isinstance().

Given that there's no real-world example underlying the example I think it wouldn't be terrible if pyright followed mypy here.

PS. If you want a response from Jukka, you probably should email him directly -- I think he's one of the many people who don't pay much attention to GitHub notifications.

[erictraut]

For what it's worth, here is the thread where this feature was discussed for TypeScript prior to deciding to move forward with it.

[erictraut]

We discussed this case in more detail with the TypeScript team. As I mentioned above, there are arguments for and against making a change here. We ultimately decided to modify the current behavior in pyright to preserve type safety in all cases even if it results in some false positive errors. We concluded that if someone is using TypedDict, they're probably already a pretty advanced Python developer — and someone who is invested in typed Python, so it's reasonable to expect that they can mark their TypedDict class as @final if they want to use this type guard form.

With this change, the code sample at the top of this bug report now generates an error within function f, but it can be avoided by making the following modification:

from typing import Union, TypedDict, final

@final
class Foo(TypedDict):
    x: int

class Bar(TypedDict):
    y: str

def f(foobar: Union[Foo, Bar]):
    if "y" in foobar:
        print(foobar["y"].lower())

[gvanrossum]

Thank you, that's a very reasonable solution.

[erictraut]

This is included in pyright 1.1.147, which I just published. It will also be included in the next release of pylance.

[ikonst]

@erictraut re your comment here, is this the change you ended up undoing in the commit you referenced?

[erictraut]

@ikonst, yes, that's correct.