Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(rust, python): handle concat_list with first lit value #7235

Merged
merged 1 commit into from
Feb 28, 2023
Merged

fix(rust, python): handle concat_list with first lit value #7235

merged 1 commit into from
Feb 28, 2023

Conversation

josh
Copy link
Contributor

@josh josh commented Feb 28, 2023

The first value passed to pl.concat_list receives some special treatment being the lhs. This means that using a pl.lit or any unit series doesn't get expanded as expected. Using a unit value in any other position works as you'd expected.

Example:

pl.DataFrame({"a": [1, 2, 3]}).select(
  pl.concat_list([pl.lit(1), pl.col("a")])
)
# ShapeError: length 3 does not match 1

Related #7236
Tangentially related #6608

@github-actions github-actions bot added fix Bug fix python Related to Python Polars rust Related to Rust Polars labels Feb 28, 2023
let max_len = other.iter().map(|s| s.len()).max().unwrap();
if max_len > 1 {
new_ca = first_ca.new_from_index(0, max_len);
first_ca = &new_ca;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a nicer Rust pattern for handling this reassignment? The compiler really wanted this new series to have a scope outside the if statement to let me reassign it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's because new_ca would get dropped when the scope ends.

You can clone on line 144. Then you can assign directly

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah! Thanks for that explanation. Still new to Rust. Does to_owned() offer any benefit over clone() there? I'm not sure how expensive a clone() on a ChunkedArray. Does it end up doing a deep copy of all it's items or just the wrapping type?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it end up doing a deep copy of all it's items or just the wrapping type?

The data is wrapped by an Arc so it is a cheap clone.

Copy link
Member

@ritchie46 ritchie46 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

let max_len = other.iter().map(|s| s.len()).max().unwrap();
if max_len > 1 {
new_ca = first_ca.new_from_index(0, max_len);
first_ca = &new_ca;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's because new_ca would get dropped when the scope ends.

You can clone on line 144. Then you can assign directly

@josh josh requested a review from ritchie46 February 28, 2023 17:10
@ritchie46 ritchie46 merged commit a7cd77f into pola-rs:master Feb 28, 2023
@josh josh deleted the concat-list-first-unit branch February 28, 2023 19:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
fix Bug fix python Related to Python Polars rust Related to Rust Polars
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants