Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf: Ensure we hit specialized gather for binary/strings #15886

Merged
merged 3 commits into from
Apr 25, 2024
Merged

Conversation

ritchie46
Copy link
Member

@ritchie46 ritchie46 commented Apr 25, 2024

Gather by slices accidentally hit the generic gather. This is very expensive for binary/string views as all buffers get naively added.

#15847

@github-actions github-actions bot added performance Performance issues or improvements python Related to Python Polars rust Related to Rust Polars labels Apr 25, 2024
@@ -23,7 +23,7 @@ impl Add for &StringChunked {
type Output = StringChunked;

fn add(self, rhs: Self) -> Self::Output {
unsafe { (self.as_binary() + rhs.as_binary()).to_string() }
unsafe { (self.as_binary() + rhs.as_binary()).to_string_unchecked() }
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

drive by. Make clear that we don't check on conversion.

Copy link

codspeed-hq bot commented Apr 25, 2024

CodSpeed Performance Report

Merging #15886 will improve performances by 20.77%

Comparing binview (a4f7159) with main (44aab96)

Summary

⚡ 2 improvements
✅ 11 untouched benchmarks

Benchmarks breakdown

Benchmark main binview Change
test_groupby_h2oai_q10 32.1 ms 26.6 ms +20.77%
test_groupby_h2oai_q2 20.6 ms 17.9 ms +15.01%

Copy link

codecov bot commented Apr 25, 2024

Codecov Report

Attention: Patch coverage is 96.15385% with 1 lines in your changes are missing coverage. Please review.

Project coverage is 81.32%. Comparing base (44aab96) to head (a4f7159).

Files Patch % Lines
...es/polars-core/src/chunked_array/arithmetic/mod.rs 50.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #15886      +/-   ##
==========================================
- Coverage   81.33%   81.32%   -0.02%     
==========================================
  Files        1385     1385              
  Lines      177003   177014      +11     
  Branches     3033     3033              
==========================================
- Hits       143971   143952      -19     
- Misses      32547    32577      +30     
  Partials      485      485              
Flag Coverage Δ
python 74.69% <92.30%> (-0.02%) ⬇️
rust 78.45% <96.15%> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@ritchie46 ritchie46 merged commit 05475da into main Apr 25, 2024
22 checks passed
@ritchie46 ritchie46 deleted the binview branch April 25, 2024 11:53
@c-peters c-peters added the accepted Ready for implementation label Apr 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
accepted Ready for implementation performance Performance issues or improvements python Related to Python Polars rust Related to Rust Polars
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

2 participants