Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

File cache invalidation not triggered for HTTP if size is the same #17683

Closed
2 tasks done
nameexhaustion opened this issue Jul 17, 2024 · 0 comments · Fixed by #17684
Closed
2 tasks done

File cache invalidation not triggered for HTTP if size is the same #17683

nameexhaustion opened this issue Jul 17, 2024 · 0 comments · Fixed by #17684
Assignees
@nameexhaustion
Labels
A-io Area: reading and writing data A-io-csv Area: reading/writing CSV files accepted Ready for implementation bug Something isn't working P-high Priority: high python Related to Python Polars

Comments

@nameexhaustion
Copy link
Collaborator

nameexhaustion commented Jul 17, 2024 •
edited
Loading

Checks

  • I have checked that this issue has not already been reported.
  • I have confirmed this bug exists on the latest version of Polars.

Reproducible example

Found when scanning a hf:// URL and updating the remote file.

Log output

No response

Issue description

We fail to detect changes if the remote file is updated but has the same size. The HTTP HEAD response is always giving a last modified timestamp of 0 - we need to use the etag instead.

Expected behavior

The updated file is downloaded.

Installed versions

1.2.0 / main
@nameexhaustion nameexhaustion added bug Something isn't working python Related to Python Polars accepted Ready for implementation P-high Priority: high A-io Area: reading and writing data A-io-csv Area: reading/writing CSV files labels Jul 17, 2024
@nameexhaustion nameexhaustion self-assigned this Jul 17, 2024
@nameexhaustion nameexhaustion mentioned this issue Jul 17, 2024
Merged
@nameexhaustion nameexhaustion changed the title File cache invalidation not triggered for HTTP unless size changes File cache invalidation not triggered for HTTP if size is the same Jul 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@nameexhaustion nameexhaustion

Labels
A-io Area: reading and writing data A-io-csv Area: reading/writing CSV files accepted Ready for implementation bug Something isn't working P-high Priority: high python Related to Python Polars
Projects
Backlog
Archived in project
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix: Use ETag for HTTP file cache invalidation nameexhaustion/polars
1 participant
@nameexhaustion