Series dtype inference based on the type of the first anyvalue

[aldanor]

Currently, it seems like when Series is constructed from any-values, it simply grabs the first non-null value and uses that to convert all other values to it? This may lead to some weird examples like:

>>> pl.Series([datetime.now(), 3.1415])  # swap these to get an error
shape: (2,)
Series: '' [datetime[μs]]
[
	2023-02-27 02:30:28.538807
	1970-01-01 00:00:00.000003   # uh, well...
]

(pandas would simply cast both to 'object')

Should it just always raise an error in cases like this? Is there are a better way to handle this kind of cases?

[ritchie46]

We should error. For pl.Series([datetime.now(), "foo"]) it errors. Somewhere an conversion should be more strict in your expample.

[aldanor]

I think it works because the primitive type for datetimes is int64 and extracting ints out of floats does work somewhere along the way.

Plus, conversion logic is (at least one branch of it) based on the type of the first non-null element, that's pretty unobvious for the user - swap the order and get different result.

[lars-reimann]

I also find the conversion logic quite confusing: In polars v0.17.6, the following code

import polars as pl
from datetime import datetime

series = pl.Series([1, True, "a", datetime.now(), None])

print(list(series))

prints [None, None, 'a', None, None]. Here it doesn't seem to be based on the type of the first element anymore but polars keeps only the strings.

On the other hand this code raises an error TypeError: 'datetime.datetime' object cannot be interpreted as an integer:

import polars as pl
from datetime import datetime

series = pl.Series([1, True, datetime.now(), "a", None])

print(list(series))

Explicitly specifying the dtype as Object makes this work as expected:

import polars as pl
from datetime import datetime

series = pl.Series([1, True, datetime.now(), "a", None], dtype=pl.Object)

print(list(series)) # [1, True, datetime.datetime(2023, 4, 22, 12, 16, 42, 870943), 'a', None]

In any case, I would expect that the description of the dtype parameter of Series "If not specified, the dtype is inferred." means that if a user omits the dtype, polars figures out something that works (like pl.Object).

[stinodego]

Closing in favor of #11156