-
Notifications
You must be signed in to change notification settings - Fork 12.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Sparse-file handling for the Linux implementation of std::fs::copy() #55909
Closed
Closed
Changes from 1 commit
Commits
Show all changes
27 commits
Select commit
Hold shift + click to select a range
52f86d2
Start of testing for fs.rs.
tarka 2e7931b
Add failing test of sparse copy.
tarka 822760a
Add some more test cases.
tarka 00e8db0
Minor test cleanups.
tarka e838cbd
Add sparse detection routine.
tarka abc69fe
Add test of current simple file copy before refactoring.
tarka b5eef91
Disable (deliberatly) non-working tests for now.
tarka c0d936a
Refactor copy_file_range() call to enable copying of chunks.
tarka e6b19ec
Break Linux copy function out to own mod for sparse refactoring.
tarka d356451
Move some internal fns up to root.
tarka 69cb124
Add functions to handle sparse files, with tests. Not used yet.
tarka 091eec7
Add tests with sparse data.
tarka be75c6f
Interim checkin; start moving kernel/uspace branch to lower-level fn.
tarka dfc92e5
Move flag of copy_file_range to thread-local, and move the decision t…
tarka 84313fb
Add override of kernel vs userspace copy for cross-device copy.
tarka 16c4010
Minor cleanup of tests.
tarka 054990f
Initial cut of user-space copy range.
tarka 95460c8
Do cross-mount detection up-front, and replace current copy impl with…
tarka b6f6f97
Enable disabled tests.
tarka 5b84746
Port in external test module as we now have a local one, and use std-…
tarka ce17903
Test and fix for userspace copies larger than read blocks.
tarka 9e52d35
Use MetadataExt rather than direct stat() call.
tarka b933836
Minor test and fmt cleanups.
tarka 92283c8
Travis runs under 3.x kernel, so we need to disable any tests that re…
tarka a437a7a
Add notes about platform-specific handling in copy().
tarka a435874
Minimise calls to .metadata().
tarka f1e7e35
Add copyright header.
tarka File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Test and fix for userspace copies larger than read blocks.
- Loading branch information
commit ce17903eccdf8bf5200b355b9554dc79001281d0
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is a reasonable default for
io::copy
that handles genericRead
andWrite
types but forfs::copy
we know we're doing syscalls and can probably do better. E.g. gnu cp has bloated up over the years and now does 128KiB or stat.blksize, whichever is larger.This is probably due to ever-growing throughput (thus more syscalls per unit of time) and larger true block sizes, e.g. on flash and raid storage.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh, and of course it only does this for large files.
On the other hand the rust std default buffer size was dropped to rust 8k in #32695 due to problems with jemalloc. But that might not be relevant here since jemalloc is not the default anymore and this would only apply to copying large files.