Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Format null arrays in Series like other series and format pl.List(pl.Null) arrays properly in dataframes. #7153

Closed
ghuls opened this issue Feb 24, 2023 · 4 comments · Fixed by #11289
Assignees
Labels
enhancement New feature or an improvement of an existing feature good first issue Good for newcomers rust Related to Rust Polars

Comments

@ghuls
Copy link
Collaborator

ghuls commented Feb 24, 2023

Problem description

Format null arrays in Series like other series and format pl.List(pl.Null) arrays properly in dataframes.

Currently:

  • Formatting of dataframe columns with pl.List(pl.Null) does not work properly.
  • A plain Polars Null series does not get printed like other series.
>>> tbl = pa.table({"a": [None, None], "b": [[None, None], [None, None]]})
>>> pl.from_arrow(tbl)
shape: (2, 2)
┌──────┬─────────────────┐
│ ab               │
│ ------             │
│ nulllist[null]      │
╞══════╪═════════════════╡
│ nullfmt implemented │
│ nullfmt implemented │
└──────┴─────────────────┘
>>> df = pl.from_arrow(tbl)
>>> df["a"]
nullarray
>>> pl.Series("a", [None, None], dtype=pl.Null)
nullarray

Compared with other (non pl.Null) Series:

>>> tbl_not_null = pa.table({"a": ["None", None], "b": [["None", None], [None, "None"]]})
>>> df = pl.from_arrow(tbl_not_null)
>>> df
shape: (2, 2)
┌──────┬────────────────┐
│ ab              │
│ ------            │
│ strlist[str]      │
╞══════╪════════════════╡
│ None ┆ ["None", null] │
│ null ┆ [null, "None"] │
└──────┴────────────────┘

>>> df["a"]
shape: (2,)
Series: 'a' [str]
[
        "None"
        null
]

>>> pl.Series("a", [None, "None"], dtype=pl.Utf8)
shape: (2,)
Series: 'a' [str]
[
        null
        "None"
]
@ghuls ghuls added good first issue Good for newcomers rust Related to Rust Polars enhancement New feature or an improvement of an existing feature labels Feb 24, 2023
@romanovacca
Copy link
Contributor

I can try to to take this up. Just started learning rust, so I might need some help on this one :)

@romanovacca
Copy link
Contributor

@ghuls can you assign this to me

@romanovacca
Copy link
Contributor

Took a bit longer to pick this up. Will look at this issue this week @ghuls

@flepknor
Copy link

flepknor commented May 11, 2023

This issue has been adressed, no?
Sorry, I only just saw that it is solved only partly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or an improvement of an existing feature good first issue Good for newcomers rust Related to Rust Polars
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants