DataFrame.join(LazyFrame) is allowed ?

[gab23r]

Research

• I have searched the above polars tags on Stack Overflow for similar questions.

• I have asked my usage related question on Stack Overflow.

Link to question on Stack Overflow

No response

Question about Polars

I think it is a new behavior but I am not sure.

I though you can't mix DataFrame and LazyFrame in join
Actually, you can if the LazyFrame is on the right side

Is it on purpose? Can we rely on this "feature" ?

import polars as pl
df = pl.DataFrame({'A': [1, 2, 3]})
ldf = pl.DataFrame({'A': [1, 2, 5]}).lazy()

_ = df.join(ldf, on='A')
# works

_ = ldf.join(df, on='A')
# ValueError: Expected a `LazyFrame` as join table, got <class 'polars.internals.dataframe.frame.DataFrame'>

[ritchie46]

That surprises me. That should not be allowed.

[ghuls]

@ritchie46 It is not so surprising as join in eager converts both dataframes to lazy anyway.

return (
            self.lazy()
            .join(
                other=other.lazy(),
                left_on=left_on,
                right_on=right_on,
                on=on,
                how=how,
                suffix=suffix,
            )
            .collect(no_optimization=True)
        )

[alexander-beedie]

My feeling is we should be explicit that the frame types are expected to match (as it might not be intentional), and raise a helpful error in all cases where they don't, telling the caller what's going on. Can make a simple PR for this, as it does seem inconsistent.

(If we were to allow it, and you actually intended to do a series of lazy computations, you may not realise you just promoted your call-chain to eager mode until you collect - or, as you now don't need to, you may forget to do that too and just eat a large performance penalty ;)