service/s3/s3manager: Add retry download object part body #843
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
xibz
suggested changes
Sep 20, 2016
| err := req.Send() | ||
| retry := 0 | ||
| var n int64 | ||
| for { |
There was a problem hiding this comment.
for retry := 0; retry < d.partBodyMaxRetries; retry++ {
}| if err != nil { | ||
| d.setErr(err) | ||
| retry++ |
There was a problem hiding this comment.
With the logic moved into the for expression, perhaps breaking early makes sense?
if err == nil {
break
}| break | ||
| } | ||
|
|
||
| d.incrWritten(n) |
There was a problem hiding this comment.
Then here upon for exit
return err| retry++ | ||
|
|
||
| // Attempt to retry failed chunk retries | ||
| if retry > d.partBodyMaxRetries { |
There was a problem hiding this comment.
Can get rid of this if block
Adds the ability to automatically retried S3 object parts that fail after the initial request response is provided. This includes scenarios such as the network connection is terminated, or broken. The body parts will be retried based on the S3 service clients Max Retries configuration. For example setting the aws.Config.MaxRetries to 3 will allow the S3 Downloader to retry part body failures up to 3 times. This feature is enabled automatically. No additional work is needed to take advantage of it. Fix aws#466
jasdel
force-pushed
the
feature/s3managerDownRetry
branch
from
7412341
to
0c5b1b9
Compare
|
Updated to switch to for loop to include retry in statement. |
xibz
approved these changes
Sep 20, 2016
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Adds the ability to automatically retried S3 object parts that fail
after the initial request response is provided. This includes scenarios
such as the network connection is terminated, or broken.
The body parts will be retried based on the S3 service clients Max
Retries configuration. For example setting the aws.Config.MaxRetries to
3 will allow the S3 Downloader to retry part body failures up to 3
times.
This feature is enabled automatically. No additional work is needed to
take advantage of it.
Fix #466