Cannot add a Series to an empty DataFrame only if the Series lengh is 1

[eitsupi]

Checks

• I have checked that this issue has not already been reported.
• I have confirmed this bug exists on the latest version of Polars.

Reproducible example

>>> import polars as pl
>>> df=pl.DataFrame()
>>> s_0=pl.Series([])
>>> s_1=pl.Series([None])
>>> s_2=pl.Series([None, None])
>>> df.with_columns(pl.lit(s_0))
shape: (0, 1)
┌──────┐
│      │
│ ---  │
│ null │
╞══════╡
└──────┘
>>> df.with_columns(pl.lit(s_1))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/usr/.local/lib/python3.10/site-packages/polars/dataframe/frame.py", line 9141, in with_columns
    return self.lazy().with_columns(*exprs, **named_exprs).collect(_eager=True)
  File "/home/usr/.local/lib/python3.10/site-packages/polars/lazyframe/frame.py", line 2032, in collect
    return wrap_df(ldf.collect(callback))
polars.exceptions.InvalidOperationError: Series , length 1 doesn't match the DataFrame height of 0

If you want this Series to be broadcasted, ensure it is a scalar (for instance by adding '.first()').
>>> df.with_columns(pl.lit(s_2))
shape: (2, 1)
┌──────┐
│      │
│ ---  │
│ null │
╞══════╡
│ null │
│ null │
└──────┘

Log output

NA

Issue description

Probably related to #18700.

When trying to add a Series as a column to an empty DataFrame, it appears to fail with an error only if the Series length is 1.

Expected behavior

It seems odd that the Series fails only if its length is 1.
It is consistent if it fails for lengths 2 or greater, or if it does not fail for length 1.

Installed versions

--------Version info---------
Polars:              1.7.1
Index type:          UInt32
Platform:            Linux-5.15.153.1-microsoft-standard-WSL2-x86_64-with-glibc2.35
Python:              3.10.12 (main, Jul 29 2024, 16:56:48) [GCC 11.4.0]

----Optional dependencies----
adbc_driver_manager  <not installed>
altair               <not installed>
cloudpickle          <not installed>
connectorx           <not installed>
deltalake            <not installed>
fastexcel            <not installed>
fsspec               <not installed>
gevent               <not installed>
great_tables         <not installed>
matplotlib           <not installed>
nest_asyncio         <not installed>
numpy                <not installed>
openpyxl             <not installed>
pandas               <not installed>
pyarrow              <not installed>
pydantic             <not installed>
pyiceberg            <not installed>
sqlalchemy           <not installed>
torch                <not installed>
xlsx2csv             <not installed>
xlsxwriter           <not installed>