Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(python): Don't modify user-supplied storage_options dict (take a shallow-copy) #15859

Merged

Conversation

alexander-beedie
Copy link
Collaborator

@alexander-beedie alexander-beedie commented Apr 24, 2024

Closes #15849.
Closes #15850.

@github-actions github-actions bot added fix Bug fix python Related to Python Polars labels Apr 24, 2024
Copy link
Member

@stinodego stinodego left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add a test for this?

Ideally, we don't always copy, but we create a copy when we add the encoding entry. But that might be micromanaging things 😬

Copy link

codecov bot commented Apr 24, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 81.37%. Comparing base (7eadccb) to head (46156a2).
Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #15859      +/-   ##
==========================================
+ Coverage   80.39%   81.37%   +0.98%     
==========================================
  Files        1263     1378     +115     
  Lines      165387   176827   +11440     
  Branches        0     3032    +3032     
==========================================
+ Hits       132957   143898   +10941     
- Misses      32430    32444      +14     
- Partials        0      485     +485     
Flag Coverage Δ
python 74.76% <100.00%> (?)
rust 78.50% <100.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@alexander-beedie
Copy link
Collaborator Author

alexander-beedie commented Apr 24, 2024

Can you add a test for this?

Done (I think; slightly hard to prove locally as I can't run the AWS tests from here, but the amended test should trigger the relevant codepath).

Ideally, we don't always copy, but we create a copy when we add the encoding entry. But that might be micromanaging things 😬

It's probably safer to do it for everything that comes into this function; otherwise we may end up having the same error again if we extend use of storage_options (and a shallow-copy is cheap for that peace of mind ;))

Copy link
Member

@stinodego stinodego left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, thanks!

I'll merge when I've fixed the CI.

@alexander-beedie alexander-beedie merged commit 3f58c8f into pola-rs:main Apr 24, 2024
13 checks passed
@alexander-beedie alexander-beedie deleted the storage-options-shallow-copy branch April 24, 2024 12:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
fix Bug fix python Related to Python Polars
Projects
None yet
2 participants