Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

upload failed when putting into s3. #449

Closed
monaka opened this issue Oct 28, 2013 · 22 comments
Closed

upload failed when putting into s3. #449

monaka opened this issue Oct 28, 2013 · 22 comments

Comments

@monaka
Copy link

monaka commented Oct 28, 2013

I have some error messages like below when I copy (upload) / sync to S3

$ aws s3 sync . s3://{bukcket-sensored}/XRaid1/backups/
upload failed: ./monacintosh-g4-1.dmg to s3://{bukcket-sensored}/XRaid1/backups/monacintosh-g4-1.dmg
HTTPSConnectionPool(host='{bukcket-sensored}.s3.amazonaws.com', port=443): Max retries exceeded with url: /XRaid1/backups/monacintosh-g4-1.dmg?partNumber=96&uploadId=1TZI4pwOOTvO4hHEJ7T0n_GtRWXyxD0fdDPeA89aJjt8lsXQ3PUSaguMQn.vbs11IOUOEDfO42DoYjs7.KiZuzcdyJ.xEUVYvcuqbLynsmm0Wbf6HluDx7iLLIYwGNJ3JdZbrnbtS1XX8F__DR2rvA-- (Caused by <class 'socket.error'>: [Errno 32] Broken pipe)
upload failed: ./monacintosh-g4-1.dmg to s3://{bukcket-sensored}/XRaid1/backups/monacintosh-g4-1.dmg
A client error (RequestTimeout) occurred: Your socket connection to the server was not read from or written to within the timeout period. Idle connections will be closed.
Completed 108 part(s) with ... file(s) remaining

{bucket-sensored} is my bucket name. Max retries exceeded are shown more than two lines.
Is it occured by my network issue?
Even if it it true, I want more thorough error messages. As I can't see what should I do to avoid the error.
@jamesls
Copy link
Member

jamesls commented Oct 28, 2013

What version of the CLI are you using? aws --version. There was an issue related to s3 retry logic that was fixed in the the latest version of the CLI.

@monaka
Copy link
Author

monaka commented Oct 28, 2013 

$ aws --version
aws-cli/1.2.1 Python/2.7.0 Darwin/10.8.0

I installed it via pip.
Should I upgrade to github HEAD revision?

@monaka
Copy link
Author

monaka commented Nov 1, 2013

After I upgrade to 1.2.3, it seems to be suppressed error messages above. I close this.

@monaka monaka closed this as completed Nov 1, 2013
@samyakbhuta
Copy link

I am encountering upload error when I try to s3 cp a large file. See below the command and what I got with --debug option set. The file in question, test.pdf is around 14MB in size. The smallest file size, which I have checked and is encountering this issue is 457 KB. I am ready to try even smaller size if that helps in debugging.

Files with 12 bytes and 804 bytes could be copied to bucket successfully. So I guess it's eventually the size that is problem here.

I am running the command from my local Macbook and not from EC2. I have DSL internet line.

command

› aws s3 cp test.pdf s3://samyakbucket --debug

command output (without --debug option set)

upload failed: ./test.pdf to s3://samyakbucket/test.pdf
HTTPSConnectionPool(host='samyakbucket.s3.amazonaws.com', port=443): Max retries exceeded with url: /test.pdf?partNumber=2&uploadId=YUYp1JYllgBQfr_lGiCzWS9XjDGncoJ38qq63JZEJWuGcyyq2rGlY5mbo3HXwmcBRc7ezkh6TEOfRmQbkfT3pkkDzT63rixYggkcNnWBb9oPK08AWVuqBDC5Ig5ePL2A (Caused by <class 'socket.error'>: [Errno 32] Broken pipe)
upload failed: ./test.pdf to s3://samyakbucket/test.pdf
HTTPSConnectionPool(host='samyakbucket.s3.amazonaws.com', port=443): Max retries exceeded with url: /test.pdf?partNumber=1&uploadId=YUYp1JYllgBQfr_lGiCzWS9XjDGncoJ38qq63JZEJWuGcyyq2rGlY5mbo3HXwmcBRc7ezkh6TEOfRmQbkfT3pkkDzT63rixYggkcNnWBb9oPK08AWVuqBDC5Ig5ePL2A (Caused by <class 'socket.error'>: [Errno 32] Broken pipe)
Completed 2 of 2 part(s) with -1 file(s) remaining

debug output

2014-01-03 15:17:44,975 - awscli.clidriver - DEBUG - CLI version: aws-cli/1.2.9 Python/2.7.6 Darwin/12.5.0, botocore version: 0.29.0
2014-01-03 15:17:44,975 - botocore.hooks - DEBUG - Event building-parameter-table.s3.cp: calling handler <function add_cmd_params at 0x101270230>
2014-01-03 15:17:44,978 - botocore.service - DEBUG - Creating service object for: s3
2014-01-03 15:17:44,978 - botocore.base - DEBUG - Attempting to load: aws/s3
2014-01-03 15:17:45,002 - botocore.base - DEBUG - Found data file: /usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/botocore/data/aws/s3.json
2014-01-03 15:17:45,002 - botocore.hooks - DEBUG - Event service-created: calling handler <function register_retries_for_service at 0x100fe5500>
2014-01-03 15:17:45,003 - botocore.handlers - DEBUG - Registering retry handlers for service: Service(s3)
2014-01-03 15:17:45,003 - botocore.hooks - DEBUG - Event creating-endpoint.s3: calling handler <function maybe_switch_to_s3sigv4 at 0x100fe55f0>
2014-01-03 15:17:45,003 - botocore.credentials - INFO - Found credentials in Environment variables.
2014-01-03 15:17:45,033 - botocore.service - DEBUG - Creating service object for: s3
2014-01-03 15:17:45,033 - botocore.hooks - DEBUG - Event service-created: calling handler <function register_retries_for_service at 0x100fe5500>
2014-01-03 15:17:45,033 - botocore.handlers - DEBUG - Registering retry handlers for service: Service(s3)
2014-01-03 15:17:45,033 - awscli.customizations.s3.executer - DEBUG - Submitting task: <awscli.customizations.s3.tasks.CreateMultipartUploadTask object at 0x101389210>
2014-01-03 15:17:45,034 - awscli.customizations.s3.executer - DEBUG - Submitting task: <awscli.customizations.s3.tasks.UploadPartTask object at 0x1013893d0>
2014-01-03 15:17:45,034 - awscli.customizations.s3.executer - DEBUG - Submitting task: <awscli.customizations.s3.tasks.UploadPartTask object at 0x101389410>
2014-01-03 15:17:45,034 - botocore.service - DEBUG - Creating service object for: s3
2014-01-03 15:17:45,034 - botocore.hooks - DEBUG - Event service-created: calling handler <function register_retries_for_service at 0x100fe5500>
2014-01-03 15:17:45,034 - botocore.handlers - DEBUG - Registering retry handlers for service: Service(s3)
2014-01-03 15:17:45,034 - awscli.customizations.s3.executer - DEBUG - Submitting task: <awscli.customizations.s3.tasks.CompleteMultipartUploadTask object at 0x101389450>
2014-01-03 15:17:45,034 - awscli.customizations.s3.tasks - DEBUG - Creating multipart upload for file: /Users/samyak/Forks/raaci/s3-uploads/forUpload/trial/test.pdf
2014-01-03 15:17:45,034 - awscli.customizations.s3.tasks - DEBUG - Uploading part 1 for filename: /Users/samyak/Forks/raaci/s3-uploads/forUpload/trial/test.pdf
2014-01-03 15:17:45,035 - awscli.customizations.s3.tasks - DEBUG - Uploading part 2 for filename: /Users/samyak/Forks/raaci/s3-uploads/forUpload/trial/test.pdf
2014-01-03 15:17:45,036 - awscli.customizations.s3.tasks - DEBUG - Waiting for upload id.
2014-01-03 15:17:45,036 - awscli.customizations.s3.tasks - DEBUG - Waiting for upload id.
2014-01-03 15:17:45,039 - awscli.customizations.s3.tasks - DEBUG - Completing multipart upload for file: /Users/samyak/Forks/raaci/s3-uploads/forUpload/trial/test.pdf
2014-01-03 15:17:45,059 - botocore.service - DEBUG - Creating operation objects for: Service(s3)
2014-01-03 15:17:45,063 - botocore.operation - DEBUG - Operation:CreateMultipartUpload called with kwargs: {'bucket': u'samyakbucket', 'key': u'test.pdf', 'content_type': 'application/pdf'}
2014-01-03 15:17:45,064 - botocore.operation - DEBUG - Creating parameter objects for: Operation:CreateMultipartUpload
2014-01-03 15:17:45,065 - botocore.endpoint - DEBUG - Making request for Operation:CreateMultipartUpload (verify_ssl=True) with params: {'headers': {u'Content-Type': 'application/pdf'}, 'uri_params': {u'Bucket': u'samyakbucket', u'Key': u'test.pdf'}, 'payload': <botocore.payload.XMLPayload object at 0x10415a290>}
2014-01-03 15:17:45,066 - botocore.endpoint - DEBUG - Building URI for rest endpoint.
2014-01-03 15:17:45,066 - botocore.endpoint - DEBUG - Templated URI path: /{Bucket}/{Key}
2014-01-03 15:17:45,066 - botocore.endpoint - DEBUG - Templated URI query_params: uploads
2014-01-03 15:17:45,066 - botocore.endpoint - DEBUG - Rendered path: /samyakbucket/test.pdf
2014-01-03 15:17:45,066 - botocore.endpoint - DEBUG - Rendered query_params: uploads
2014-01-03 15:17:45,066 - botocore.hooks - DEBUG - Event before-auth.s3: calling handler <function fix_s3_host at 0x100fe5410>
2014-01-03 15:17:45,066 - botocore.handlers - DEBUG - Checking for DNS compatible bucket for: https://s3-us-west-1.amazonaws.com/samyakbucket/test.pdf?uploads
2014-01-03 15:17:45,066 - botocore.handlers - DEBUG - URI updated to: https://samyakbucket.s3.amazonaws.com/test.pdf?uploads
2014-01-03 15:17:45,067 - botocore.auth - DEBUG - Calculating signature using hmacv1 auth.
2014-01-03 15:17:45,067 - botocore.auth - DEBUG - HTTP request method: POST
2014-01-03 15:17:45,067 - botocore.auth - DEBUG - StringToSign:
POST

application/pdf
Fri, 03 Jan 2014 09:47:45 GMT
/samyakbucket/test.pdf?uploads
2014-01-03 15:17:45,097 - botocore.endpoint - DEBUG - Sending http request: <PreparedRequest [POST]>
2014-01-03 15:17:45,098 - botocore.vendored.requests.packages.urllib3.connectionpool - INFO - Starting new HTTPS connection (1): samyakbucket.s3.amazonaws.com
2014-01-03 15:17:47,321 - botocore.vendored.requests.packages.urllib3.connectionpool - DEBUG - "POST /test.pdf?uploads HTTP/1.1" 307 None
2014-01-03 15:17:47,322 - botocore.vendored.requests.packages.urllib3.connectionpool - INFO - Starting new HTTPS connection (1): samyakbucket.s3-us-west-1.amazonaws.com
2014-01-03 15:17:49,588 - botocore.vendored.requests.packages.urllib3.connectionpool - DEBUG - "POST /test.pdf?uploads HTTP/1.1" 200 None
2014-01-03 15:17:49,589 - botocore.response - DEBUG - Response Body:
<?xml version="1.0" encoding="UTF-8"?>
<InitiateMultipartUploadResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Bucket>samyakbucket</Bucket><Key>test.pdf</Key><UploadId>YUYp1JYllgBQfr_lGiCzWS9XjDGncoJ38qq63JZEJWuGcyyq2rGlY5mbo3HXwmcBRc7ezkh6TEOfRmQbkfT3pkkDzT63rixYggkcNnWBb9oPK08AWVuqBDC5Ig5ePL2A</UploadId></InitiateMultipartUploadResult>
2014-01-03 15:17:49,589 - botocore.hooks - DEBUG - Event needs-retry.s3.CreateMultipartUpload: calling handler <botocore.retryhandler.RetryHandler object at 0x10129bcd0>
2014-01-03 15:17:49,589 - botocore.retryhandler - DEBUG - No retry needed.
2014-01-03 15:17:49,589 - botocore.hooks - DEBUG - Event after-call.s3.CreateMultipartUpload: calling handler <awscli.errorhandler.ErrorHandler object at 0x1012889d0>
2014-01-03 15:17:49,589 - awscli.errorhandler - DEBUG - HTTP Response Code: 200
2014-01-03 15:17:49,590 - awscli.customizations.s3.tasks - DEBUG - Announcing upload id: YUYp1JYllgBQfr_lGiCzWS9XjDGncoJ38qq63JZEJWuGcyyq2rGlY5mbo3HXwmcBRc7ezkh6TEOfRmQbkfT3pkkDzT63rixYggkcNnWBb9oPK08AWVuqBDC5Ig5ePL2A
2014-01-03 15:17:49,616 - botocore.operation - DEBUG - Operation:UploadPart called with kwargs: {'body': <awscli.customizations.s3.utils.ReadFileChunk object at 0x10415a290>, 'upload_id': 'YUYp1JYllgBQfr_lGiCzWS9XjDGncoJ38qq63JZEJWuGcyyq2rGlY5mbo3HXwmcBRc7ezkh6TEOfRmQbkfT3pkkDzT63rixYggkcNnWBb9oPK08AWVuqBDC5Ig5ePL2A', 'part_number': 1, 'bucket': u'samyakbucket', 'key': u'test.pdf'}
2014-01-03 15:17:49,617 - botocore.operation - DEBUG - Creating parameter objects for: Operation:UploadPart
2014-01-03 15:17:49,617 - botocore.operation - DEBUG - Operation:UploadPart called with kwargs: {'body': <awscli.customizations.s3.utils.ReadFileChunk object at 0x1041644d0>, 'upload_id': 'YUYp1JYllgBQfr_lGiCzWS9XjDGncoJ38qq63JZEJWuGcyyq2rGlY5mbo3HXwmcBRc7ezkh6TEOfRmQbkfT3pkkDzT63rixYggkcNnWBb9oPK08AWVuqBDC5Ig5ePL2A', 'part_number': 2, 'bucket': u'samyakbucket', 'key': u'test.pdf'}
2014-01-03 15:17:49,618 - botocore.endpoint - DEBUG - Making request for Operation:UploadPart (verify_ssl=True) with params: {'headers': {}, 'uri_params': {u'UploadId': 'YUYp1JYllgBQfr_lGiCzWS9XjDGncoJ38qq63JZEJWuGcyyq2rGlY5mbo3HXwmcBRc7ezkh6TEOfRmQbkfT3pkkDzT63rixYggkcNnWBb9oPK08AWVuqBDC5Ig5ePL2A', u'PartNumber': 1, u'Bucket': u'samyakbucket', u'Key': u'test.pdf'}, 'payload': <botocore.payload.Payload object at 0x104164810>}
2014-01-03 15:17:49,618 - botocore.endpoint - DEBUG - Making request for Operation:UploadPart (verify_ssl=True) with params: {'headers': {}, 'uri_params': {u'UploadId': 'YUYp1JYllgBQfr_lGiCzWS9XjDGncoJ38qq63JZEJWuGcyyq2rGlY5mbo3HXwmcBRc7ezkh6TEOfRmQbkfT3pkkDzT63rixYggkcNnWBb9oPK08AWVuqBDC5Ig5ePL2A', u'PartNumber': 2, u'Bucket': u'samyakbucket', u'Key': u'test.pdf'}, 'payload': <botocore.payload.Payload object at 0x104164490>}
2014-01-03 15:17:49,618 - botocore.endpoint - DEBUG - Building URI for rest endpoint.
2014-01-03 15:17:49,618 - botocore.endpoint - DEBUG - Building URI for rest endpoint.
2014-01-03 15:17:49,618 - botocore.endpoint - DEBUG - Templated URI path: /{Bucket}/{Key}
2014-01-03 15:17:49,618 - botocore.endpoint - DEBUG - Templated URI path: /{Bucket}/{Key}
2014-01-03 15:17:49,619 - botocore.endpoint - DEBUG - Templated URI query_params: partNumber={PartNumber}&uploadId={UploadId}
2014-01-03 15:17:49,619 - botocore.endpoint - DEBUG - Templated URI query_params: partNumber={PartNumber}&uploadId={UploadId}
2014-01-03 15:17:49,619 - botocore.endpoint - DEBUG - Rendered path: /samyakbucket/test.pdf
2014-01-03 15:17:49,619 - botocore.endpoint - DEBUG - Rendered path: /samyakbucket/test.pdf
2014-01-03 15:17:49,619 - botocore.endpoint - DEBUG - Rendered query_params: partNumber=1&uploadId=YUYp1JYllgBQfr_lGiCzWS9XjDGncoJ38qq63JZEJWuGcyyq2rGlY5mbo3HXwmcBRc7ezkh6TEOfRmQbkfT3pkkDzT63rixYggkcNnWBb9oPK08AWVuqBDC5Ig5ePL2A
2014-01-03 15:17:49,620 - botocore.endpoint - DEBUG - Rendered query_params: partNumber=2&uploadId=YUYp1JYllgBQfr_lGiCzWS9XjDGncoJ38qq63JZEJWuGcyyq2rGlY5mbo3HXwmcBRc7ezkh6TEOfRmQbkfT3pkkDzT63rixYggkcNnWBb9oPK08AWVuqBDC5Ig5ePL2A
2014-01-03 15:17:49,620 - botocore.hooks - DEBUG - Event before-auth.s3: calling handler <function fix_s3_host at 0x100fe5410>
2014-01-03 15:17:49,620 - botocore.handlers - DEBUG - Checking for DNS compatible bucket for: https://s3-us-west-1.amazonaws.com/samyakbucket/test.pdf?partNumber=1&uploadId=YUYp1JYllgBQfr_lGiCzWS9XjDGncoJ38qq63JZEJWuGcyyq2rGlY5mbo3HXwmcBRc7ezkh6TEOfRmQbkfT3pkkDzT63rixYggkcNnWBb9oPK08AWVuqBDC5Ig5ePL2A
2014-01-03 15:17:49,620 - botocore.handlers - DEBUG - URI updated to: https://samyakbucket.s3.amazonaws.com/test.pdf?partNumber=1&uploadId=YUYp1JYllgBQfr_lGiCzWS9XjDGncoJ38qq63JZEJWuGcyyq2rGlY5mbo3HXwmcBRc7ezkh6TEOfRmQbkfT3pkkDzT63rixYggkcNnWBb9oPK08AWVuqBDC5Ig5ePL2A
2014-01-03 15:17:49,620 - botocore.auth - DEBUG - Calculating signature using hmacv1 auth.
2014-01-03 15:17:49,621 - botocore.auth - DEBUG - HTTP request method: PUT
2014-01-03 15:17:49,621 - botocore.auth - DEBUG - StringToSign:
PUT


Fri, 03 Jan 2014 09:47:49 GMT
/samyakbucket/test.pdf?partNumber=1&uploadId=YUYp1JYllgBQfr_lGiCzWS9XjDGncoJ38qq63JZEJWuGcyyq2rGlY5mbo3HXwmcBRc7ezkh6TEOfRmQbkfT3pkkDzT63rixYggkcNnWBb9oPK08AWVuqBDC5Ig5ePL2A
2014-01-03 15:17:49,621 - botocore.endpoint - DEBUG - Sending http request: <PreparedRequest [PUT]>
2014-01-03 15:17:49,621 - botocore.hooks - DEBUG - Event before-auth.s3: calling handler <function fix_s3_host at 0x100fe5410>
2014-01-03 15:17:49,622 - botocore.vendored.requests.packages.urllib3.connectionpool - INFO - Resetting dropped connection: samyakbucket.s3.amazonaws.com
2014-01-03 15:17:49,622 - botocore.handlers - DEBUG - Checking for DNS compatible bucket for: https://s3-us-west-1.amazonaws.com/samyakbucket/test.pdf?partNumber=2&uploadId=YUYp1JYllgBQfr_lGiCzWS9XjDGncoJ38qq63JZEJWuGcyyq2rGlY5mbo3HXwmcBRc7ezkh6TEOfRmQbkfT3pkkDzT63rixYggkcNnWBb9oPK08AWVuqBDC5Ig5ePL2A
2014-01-03 15:17:49,624 - botocore.handlers - DEBUG - URI updated to: https://samyakbucket.s3.amazonaws.com/test.pdf?partNumber=2&uploadId=YUYp1JYllgBQfr_lGiCzWS9XjDGncoJ38qq63JZEJWuGcyyq2rGlY5mbo3HXwmcBRc7ezkh6TEOfRmQbkfT3pkkDzT63rixYggkcNnWBb9oPK08AWVuqBDC5Ig5ePL2A
2014-01-03 15:17:49,625 - botocore.auth - DEBUG - Calculating signature using hmacv1 auth.
2014-01-03 15:17:49,625 - botocore.auth - DEBUG - HTTP request method: PUT
2014-01-03 15:17:49,625 - botocore.auth - DEBUG - StringToSign:
PUT


Fri, 03 Jan 2014 09:47:49 GMT
/samyakbucket/test.pdf?partNumber=2&uploadId=YUYp1JYllgBQfr_lGiCzWS9XjDGncoJ38qq63JZEJWuGcyyq2rGlY5mbo3HXwmcBRc7ezkh6TEOfRmQbkfT3pkkDzT63rixYggkcNnWBb9oPK08AWVuqBDC5Ig5ePL2A
2014-01-03 15:17:49,685 - botocore.endpoint - DEBUG - Sending http request: <PreparedRequest [PUT]>
2014-01-03 15:17:49,685 - botocore.vendored.requests.packages.urllib3.connectionpool - INFO - Starting new HTTPS connection (2): samyakbucket.s3.amazonaws.com
2014-01-03 15:17:52,181 - botocore.hooks - DEBUG - Event needs-retry.s3.UploadPart: calling handler <botocore.retryhandler.RetryHandler object at 0x10129bcd0>
2014-01-03 15:17:52,181 - botocore.retryhandler - DEBUG - retry needed, retryable exception caught: HTTPSConnectionPool(host='samyakbucket.s3.amazonaws.com', port=443): Max retries exceeded with url: /test.pdf?partNumber=1&uploadId=YUYp1JYllgBQfr_lGiCzWS9XjDGncoJ38qq63JZEJWuGcyyq2rGlY5mbo3HXwmcBRc7ezkh6TEOfRmQbkfT3pkkDzT63rixYggkcNnWBb9oPK08AWVuqBDC5Ig5ePL2A (Caused by <class 'socket.error'>: [Errno 32] Broken pipe)
Traceback (most recent call last):
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/botocore/retryhandler.py", line 270, in _should_retry
    return self._checker(attempt_number, response, caught_exception)
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/botocore/retryhandler.py", line 318, in __call__
    caught_exception)
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/botocore/retryhandler.py", line 227, in __call__
    return self._check_caught_exception(attempt_number, caught_exception)
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/botocore/retryhandler.py", line 360, in _check_caught_exception
    raise caught_exception
ConnectionError: HTTPSConnectionPool(host='samyakbucket.s3.amazonaws.com', port=443): Max retries exceeded with url: /test.pdf?partNumber=1&uploadId=YUYp1JYllgBQfr_lGiCzWS9XjDGncoJ38qq63JZEJWuGcyyq2rGlY5mbo3HXwmcBRc7ezkh6TEOfRmQbkfT3pkkDzT63rixYggkcNnWBb9oPK08AWVuqBDC5Ig5ePL2A (Caused by <class 'socket.error'>: [Errno 32] Broken pipe)
2014-01-03 15:17:52,219 - botocore.retryhandler - DEBUG - Retry needed, action of: 0.651018168309
2014-01-03 15:17:52,219 - botocore.endpoint - DEBUG - Response received to retry, sleeping for 0.651018168309 seconds
2014-01-03 15:17:52,776 - botocore.hooks - DEBUG - Event needs-retry.s3.UploadPart: calling handler <botocore.retryhandler.RetryHandler object at 0x10129bcd0>
2014-01-03 15:17:52,776 - botocore.retryhandler - DEBUG - retry needed, retryable exception caught: HTTPSConnectionPool(host='samyakbucket.s3.amazonaws.com', port=443): Max retries exceeded with url: /test.pdf?partNumber=2&uploadId=YUYp1JYllgBQfr_lGiCzWS9XjDGncoJ38qq63JZEJWuGcyyq2rGlY5mbo3HXwmcBRc7ezkh6TEOfRmQbkfT3pkkDzT63rixYggkcNnWBb9oPK08AWVuqBDC5Ig5ePL2A (Caused by <class 'socket.error'>: [Errno 32] Broken pipe)
Traceback (most recent call last):
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/botocore/retryhandler.py", line 270, in _should_retry
    return self._checker(attempt_number, response, caught_exception)
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/botocore/retryhandler.py", line 318, in __call__
    caught_exception)
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/botocore/retryhandler.py", line 227, in __call__
    return self._check_caught_exception(attempt_number, caught_exception)
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/botocore/retryhandler.py", line 360, in _check_caught_exception
    raise caught_exception
ConnectionError: HTTPSConnectionPool(host='samyakbucket.s3.amazonaws.com', port=443): Max retries exceeded with url: /test.pdf?partNumber=2&uploadId=YUYp1JYllgBQfr_lGiCzWS9XjDGncoJ38qq63JZEJWuGcyyq2rGlY5mbo3HXwmcBRc7ezkh6TEOfRmQbkfT3pkkDzT63rixYggkcNnWBb9oPK08AWVuqBDC5Ig5ePL2A (Caused by <class 'socket.error'>: [Errno 32] Broken pipe)
2014-01-03 15:17:52,776 - botocore.retryhandler - DEBUG - Retry needed, action of: 0.0333211428092
2014-01-03 15:17:52,776 - botocore.endpoint - DEBUG - Response received to retry, sleeping for 0.0333211428092 seconds
2014-01-03 15:17:52,811 - botocore.awsrequest - DEBUG - Rewinding stream: <awscli.customizations.s3.utils.ReadFileChunk object at 0x1041644d0>
2014-01-03 15:17:52,811 - botocore.endpoint - DEBUG - Sending http request: <PreparedRequest [PUT]>
2014-01-03 15:17:52,812 - botocore.vendored.requests.packages.urllib3.connectionpool - INFO - Starting new HTTPS connection (3): samyakbucket.s3.amazonaws.com
2014-01-03 15:17:52,871 - botocore.awsrequest - DEBUG - Rewinding stream: <awscli.customizations.s3.utils.ReadFileChunk object at 0x10415a290>
2014-01-03 15:17:52,871 - botocore.endpoint - DEBUG - Sending http request: <PreparedRequest [PUT]>
2014-01-03 15:17:52,872 - botocore.vendored.requests.packages.urllib3.connectionpool - INFO - Starting new HTTPS connection (4): samyakbucket.s3.amazonaws.com
2014-01-03 15:17:56,128 - botocore.hooks - DEBUG - Event needs-retry.s3.UploadPart: calling handler <botocore.retryhandler.RetryHandler object at 0x10129bcd0>
2014-01-03 15:17:56,128 - botocore.retryhandler - DEBUG - retry needed, retryable exception caught: HTTPSConnectionPool(host='samyakbucket.s3.amazonaws.com', port=443): Max retries exceeded with url: /test.pdf?partNumber=2&uploadId=YUYp1JYllgBQfr_lGiCzWS9XjDGncoJ38qq63JZEJWuGcyyq2rGlY5mbo3HXwmcBRc7ezkh6TEOfRmQbkfT3pkkDzT63rixYggkcNnWBb9oPK08AWVuqBDC5Ig5ePL2A (Caused by <class 'socket.error'>: [Errno 32] Broken pipe)
Traceback (most recent call last):
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/botocore/retryhandler.py", line 270, in _should_retry
    return self._checker(attempt_number, response, caught_exception)
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/botocore/retryhandler.py", line 318, in __call__
    caught_exception)
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/botocore/retryhandler.py", line 227, in __call__
    return self._check_caught_exception(attempt_number, caught_exception)
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/botocore/retryhandler.py", line 360, in _check_caught_exception
    raise caught_exception
ConnectionError: HTTPSConnectionPool(host='samyakbucket.s3.amazonaws.com', port=443): Max retries exceeded with url: /test.pdf?partNumber=2&uploadId=YUYp1JYllgBQfr_lGiCzWS9XjDGncoJ38qq63JZEJWuGcyyq2rGlY5mbo3HXwmcBRc7ezkh6TEOfRmQbkfT3pkkDzT63rixYggkcNnWBb9oPK08AWVuqBDC5Ig5ePL2A (Caused by <class 'socket.error'>: [Errno 32] Broken pipe)
2014-01-03 15:17:56,129 - botocore.retryhandler - DEBUG - Retry needed, action of: 1.62846071451
2014-01-03 15:17:56,129 - botocore.endpoint - DEBUG - Response received to retry, sleeping for 1.62846071451 seconds
2014-01-03 15:17:56,687 - botocore.hooks - DEBUG - Event needs-retry.s3.UploadPart: calling handler <botocore.retryhandler.RetryHandler object at 0x10129bcd0>
2014-01-03 15:17:56,688 - botocore.retryhandler - DEBUG - retry needed, retryable exception caught: HTTPSConnectionPool(host='samyakbucket.s3.amazonaws.com', port=443): Max retries exceeded with url: /test.pdf?partNumber=1&uploadId=YUYp1JYllgBQfr_lGiCzWS9XjDGncoJ38qq63JZEJWuGcyyq2rGlY5mbo3HXwmcBRc7ezkh6TEOfRmQbkfT3pkkDzT63rixYggkcNnWBb9oPK08AWVuqBDC5Ig5ePL2A (Caused by <class 'socket.error'>: [Errno 32] Broken pipe)
Traceback (most recent call last):
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/botocore/retryhandler.py", line 270, in _should_retry
    return self._checker(attempt_number, response, caught_exception)
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/botocore/retryhandler.py", line 318, in __call__
    caught_exception)
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/botocore/retryhandler.py", line 227, in __call__
    return self._check_caught_exception(attempt_number, caught_exception)
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/botocore/retryhandler.py", line 360, in _check_caught_exception
    raise caught_exception
ConnectionError: HTTPSConnectionPool(host='samyakbucket.s3.amazonaws.com', port=443): Max retries exceeded with url: /test.pdf?partNumber=1&uploadId=YUYp1JYllgBQfr_lGiCzWS9XjDGncoJ38qq63JZEJWuGcyyq2rGlY5mbo3HXwmcBRc7ezkh6TEOfRmQbkfT3pkkDzT63rixYggkcNnWBb9oPK08AWVuqBDC5Ig5ePL2A (Caused by <class 'socket.error'>: [Errno 32] Broken pipe)
2014-01-03 15:17:56,688 - botocore.retryhandler - DEBUG - Retry needed, action of: 1.98296193968
2014-01-03 15:17:56,688 - botocore.endpoint - DEBUG - Response received to retry, sleeping for 1.98296193968 seconds
2014-01-03 15:17:57,758 - botocore.awsrequest - DEBUG - Rewinding stream: <awscli.customizations.s3.utils.ReadFileChunk object at 0x1041644d0>
2014-01-03 15:17:57,759 - botocore.endpoint - DEBUG - Sending http request: <PreparedRequest [PUT]>
2014-01-03 15:17:57,759 - botocore.vendored.requests.packages.urllib3.connectionpool - INFO - Starting new HTTPS connection (5): samyakbucket.s3.amazonaws.com
2014-01-03 15:17:58,671 - botocore.awsrequest - DEBUG - Rewinding stream: <awscli.customizations.s3.utils.ReadFileChunk object at 0x10415a290>
2014-01-03 15:17:58,672 - botocore.endpoint - DEBUG - Sending http request: <PreparedRequest [PUT]>
2014-01-03 15:17:58,672 - botocore.vendored.requests.packages.urllib3.connectionpool - INFO - Starting new HTTPS connection (6): samyakbucket.s3.amazonaws.com
2014-01-03 15:18:00,333 - botocore.hooks - DEBUG - Event needs-retry.s3.UploadPart: calling handler <botocore.retryhandler.RetryHandler object at 0x10129bcd0>
2014-01-03 15:18:00,333 - botocore.retryhandler - DEBUG - retry needed, retryable exception caught: HTTPSConnectionPool(host='samyakbucket.s3.amazonaws.com', port=443): Max retries exceeded with url: /test.pdf?partNumber=2&uploadId=YUYp1JYllgBQfr_lGiCzWS9XjDGncoJ38qq63JZEJWuGcyyq2rGlY5mbo3HXwmcBRc7ezkh6TEOfRmQbkfT3pkkDzT63rixYggkcNnWBb9oPK08AWVuqBDC5Ig5ePL2A (Caused by <class 'socket.error'>: [Errno 32] Broken pipe)
Traceback (most recent call last):
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/botocore/retryhandler.py", line 270, in _should_retry
    return self._checker(attempt_number, response, caught_exception)
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/botocore/retryhandler.py", line 318, in __call__
    caught_exception)
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/botocore/retryhandler.py", line 227, in __call__
    return self._check_caught_exception(attempt_number, caught_exception)
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/botocore/retryhandler.py", line 360, in _check_caught_exception
    raise caught_exception
ConnectionError: HTTPSConnectionPool(host='samyakbucket.s3.amazonaws.com', port=443): Max retries exceeded with url: /test.pdf?partNumber=2&uploadId=YUYp1JYllgBQfr_lGiCzWS9XjDGncoJ38qq63JZEJWuGcyyq2rGlY5mbo3HXwmcBRc7ezkh6TEOfRmQbkfT3pkkDzT63rixYggkcNnWBb9oPK08AWVuqBDC5Ig5ePL2A (Caused by <class 'socket.error'>: [Errno 32] Broken pipe)
2014-01-03 15:18:00,334 - botocore.retryhandler - DEBUG - Retry needed, action of: 1.43535438794
2014-01-03 15:18:00,334 - botocore.endpoint - DEBUG - Response received to retry, sleeping for 1.43535438794 seconds
2014-01-03 15:18:01,770 - botocore.awsrequest - DEBUG - Rewinding stream: <awscli.customizations.s3.utils.ReadFileChunk object at 0x1041644d0>
2014-01-03 15:18:01,770 - botocore.endpoint - DEBUG - Sending http request: <PreparedRequest [PUT]>
2014-01-03 15:18:01,771 - botocore.vendored.requests.packages.urllib3.connectionpool - INFO - Starting new HTTPS connection (7): samyakbucket.s3.amazonaws.com
2014-01-03 15:18:02,461 - botocore.hooks - DEBUG - Event needs-retry.s3.UploadPart: calling handler <botocore.retryhandler.RetryHandler object at 0x10129bcd0>
2014-01-03 15:18:02,461 - botocore.retryhandler - DEBUG - retry needed, retryable exception caught: HTTPSConnectionPool(host='samyakbucket.s3.amazonaws.com', port=443): Max retries exceeded with url: /test.pdf?partNumber=1&uploadId=YUYp1JYllgBQfr_lGiCzWS9XjDGncoJ38qq63JZEJWuGcyyq2rGlY5mbo3HXwmcBRc7ezkh6TEOfRmQbkfT3pkkDzT63rixYggkcNnWBb9oPK08AWVuqBDC5Ig5ePL2A (Caused by <class 'socket.error'>: [Errno 32] Broken pipe)
Traceback (most recent call last):
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/botocore/retryhandler.py", line 270, in _should_retry
    return self._checker(attempt_number, response, caught_exception)
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/botocore/retryhandler.py", line 318, in __call__
    caught_exception)
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/botocore/retryhandler.py", line 227, in __call__
    return self._check_caught_exception(attempt_number, caught_exception)
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/botocore/retryhandler.py", line 360, in _check_caught_exception
    raise caught_exception
ConnectionError: HTTPSConnectionPool(host='samyakbucket.s3.amazonaws.com', port=443): Max retries exceeded with url: /test.pdf?partNumber=1&uploadId=YUYp1JYllgBQfr_lGiCzWS9XjDGncoJ38qq63JZEJWuGcyyq2rGlY5mbo3HXwmcBRc7ezkh6TEOfRmQbkfT3pkkDzT63rixYggkcNnWBb9oPK08AWVuqBDC5Ig5ePL2A (Caused by <class 'socket.error'>: [Errno 32] Broken pipe)
2014-01-03 15:18:02,462 - botocore.retryhandler - DEBUG - Retry needed, action of: 3.11774050011
2014-01-03 15:18:02,462 - botocore.endpoint - DEBUG - Response received to retry, sleeping for 3.11774050011 seconds
2014-01-03 15:18:05,128 - botocore.hooks - DEBUG - Event needs-retry.s3.UploadPart: calling handler <botocore.retryhandler.RetryHandler object at 0x10129bcd0>
2014-01-03 15:18:05,129 - botocore.retryhandler - DEBUG - retry needed, retryable exception caught: HTTPSConnectionPool(host='samyakbucket.s3.amazonaws.com', port=443): Max retries exceeded with url: /test.pdf?partNumber=2&uploadId=YUYp1JYllgBQfr_lGiCzWS9XjDGncoJ38qq63JZEJWuGcyyq2rGlY5mbo3HXwmcBRc7ezkh6TEOfRmQbkfT3pkkDzT63rixYggkcNnWBb9oPK08AWVuqBDC5Ig5ePL2A (Caused by <class 'socket.error'>: [Errno 32] Broken pipe)
Traceback (most recent call last):
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/botocore/retryhandler.py", line 270, in _should_retry
    return self._checker(attempt_number, response, caught_exception)
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/botocore/retryhandler.py", line 318, in __call__
    caught_exception)
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/botocore/retryhandler.py", line 227, in __call__
    return self._check_caught_exception(attempt_number, caught_exception)
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/botocore/retryhandler.py", line 360, in _check_caught_exception
    raise caught_exception
ConnectionError: HTTPSConnectionPool(host='samyakbucket.s3.amazonaws.com', port=443): Max retries exceeded with url: /test.pdf?partNumber=2&uploadId=YUYp1JYllgBQfr_lGiCzWS9XjDGncoJ38qq63JZEJWuGcyyq2rGlY5mbo3HXwmcBRc7ezkh6TEOfRmQbkfT3pkkDzT63rixYggkcNnWBb9oPK08AWVuqBDC5Ig5ePL2A (Caused by <class 'socket.error'>: [Errno 32] Broken pipe)
2014-01-03 15:18:05,147 - botocore.retryhandler - DEBUG - Retry needed, action of: 5.28398908296
2014-01-03 15:18:05,147 - botocore.endpoint - DEBUG - Response received to retry, sleeping for 5.28398908296 seconds
2014-01-03 15:18:05,580 - botocore.awsrequest - DEBUG - Rewinding stream: <awscli.customizations.s3.utils.ReadFileChunk object at 0x10415a290>
2014-01-03 15:18:05,580 - botocore.endpoint - DEBUG - Sending http request: <PreparedRequest [PUT]>
2014-01-03 15:18:05,581 - botocore.vendored.requests.packages.urllib3.connectionpool - INFO - Starting new HTTPS connection (8): samyakbucket.s3.amazonaws.com
2014-01-03 15:18:08,426 - botocore.hooks - DEBUG - Event needs-retry.s3.UploadPart: calling handler <botocore.retryhandler.RetryHandler object at 0x10129bcd0>
2014-01-03 15:18:08,427 - botocore.retryhandler - DEBUG - retry needed, retryable exception caught: HTTPSConnectionPool(host='samyakbucket.s3.amazonaws.com', port=443): Max retries exceeded with url: /test.pdf?partNumber=1&uploadId=YUYp1JYllgBQfr_lGiCzWS9XjDGncoJ38qq63JZEJWuGcyyq2rGlY5mbo3HXwmcBRc7ezkh6TEOfRmQbkfT3pkkDzT63rixYggkcNnWBb9oPK08AWVuqBDC5Ig5ePL2A (Caused by <class 'socket.error'>: [Errno 32] Broken pipe)
Traceback (most recent call last):
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/botocore/retryhandler.py", line 270, in _should_retry
    return self._checker(attempt_number, response, caught_exception)
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/botocore/retryhandler.py", line 318, in __call__
    caught_exception)
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/botocore/retryhandler.py", line 227, in __call__
    return self._check_caught_exception(attempt_number, caught_exception)
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/botocore/retryhandler.py", line 360, in _check_caught_exception
    raise caught_exception
ConnectionError: HTTPSConnectionPool(host='samyakbucket.s3.amazonaws.com', port=443): Max retries exceeded with url: /test.pdf?partNumber=1&uploadId=YUYp1JYllgBQfr_lGiCzWS9XjDGncoJ38qq63JZEJWuGcyyq2rGlY5mbo3HXwmcBRc7ezkh6TEOfRmQbkfT3pkkDzT63rixYggkcNnWBb9oPK08AWVuqBDC5Ig5ePL2A (Caused by <class 'socket.error'>: [Errno 32] Broken pipe)
2014-01-03 15:18:08,427 - botocore.retryhandler - DEBUG - Retry needed, action of: 3.64608242904
2014-01-03 15:18:08,427 - botocore.endpoint - DEBUG - Response received to retry, sleeping for 3.64608242904 seconds
2014-01-03 15:18:10,432 - botocore.awsrequest - DEBUG - Rewinding stream: <awscli.customizations.s3.utils.ReadFileChunk object at 0x1041644d0>
2014-01-03 15:18:10,432 - botocore.endpoint - DEBUG - Sending http request: <PreparedRequest [PUT]>
2014-01-03 15:18:10,433 - botocore.vendored.requests.packages.urllib3.connectionpool - INFO - Starting new HTTPS connection (9): samyakbucket.s3.amazonaws.com
2014-01-03 15:18:12,074 - botocore.awsrequest - DEBUG - Rewinding stream: <awscli.customizations.s3.utils.ReadFileChunk object at 0x10415a290>
2014-01-03 15:18:12,075 - botocore.endpoint - DEBUG - Sending http request: <PreparedRequest [PUT]>
2014-01-03 15:18:12,075 - botocore.vendored.requests.packages.urllib3.connectionpool - INFO - Starting new HTTPS connection (10): samyakbucket.s3.amazonaws.com
2014-01-03 15:18:13,009 - botocore.hooks - DEBUG - Event needs-retry.s3.UploadPart: calling handler <botocore.retryhandler.RetryHandler object at 0x10129bcd0>
2014-01-03 15:18:13,010 - awscli.customizations.s3.tasks - DEBUG - Error during part upload: HTTPSConnectionPool(host='samyakbucket.s3.amazonaws.com', port=443): Max retries exceeded with url: /test.pdf?partNumber=2&uploadId=YUYp1JYllgBQfr_lGiCzWS9XjDGncoJ38qq63JZEJWuGcyyq2rGlY5mbo3HXwmcBRc7ezkh6TEOfRmQbkfT3pkkDzT63rixYggkcNnWBb9oPK08AWVuqBDC5Ig5ePL2A (Caused by <class 'socket.error'>: [Errno 32] Broken pipe)
Traceback (most recent call last):
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/awscli/customizations/s3/tasks.py", line 217, in __call__
    self._filename.service, 'UploadPart', params)
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/awscli/customizations/s3/utils.py", line 128, in operate
    http_response, response_data = operation.call(**kwargs)
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/botocore/operation.py", line 76, in call
    response = endpoint.make_request(self, params)
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/botocore/endpoint.py", line 75, in make_request
    return self._send_request(prepared_request, operation)
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/botocore/endpoint.py", line 97, in _send_request
    while self._needs_retry(attempts, operation, response, exception):
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/botocore/endpoint.py", line 128, in _needs_retry
    caught_exception=caught_exception)
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/botocore/session.py", line 554, in emit_first_non_none_response
    responses = self._events.emit(event_name, **kwargs)
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/botocore/hooks.py", line 158, in emit
    response = handler(**kwargs)
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/botocore/retryhandler.py", line 188, in __call__
    if self._checker(attempts, response, caught_exception):
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/botocore/retryhandler.py", line 255, in __call__
    caught_exception)
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/botocore/retryhandler.py", line 278, in _should_retry
    return self._checker(attempt_number, response, caught_exception)
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/botocore/retryhandler.py", line 318, in __call__
    caught_exception)
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/botocore/retryhandler.py", line 227, in __call__
    return self._check_caught_exception(attempt_number, caught_exception)
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/botocore/retryhandler.py", line 360, in _check_caught_exception
    raise caught_exception
ConnectionError: HTTPSConnectionPool(host='samyakbucket.s3.amazonaws.com', port=443): Max retries exceeded with url: /test.pdf?partNumber=2&uploadId=YUYp1JYllgBQfr_lGiCzWS9XjDGncoJ38qq63JZEJWuGcyyq2rGlY5mbo3HXwmcBRc7ezkh6TEOfRmQbkfT3pkkDzT63rixYggkcNnWBb9oPK08AWVuqBDC5Ig5ePL2A (Caused by <class 'socket.error'>: [Errno 32] Broken pipe)
2014-01-03 15:18:13,124 - awscli.customizations.s3.executer - DEBUG - Received print task: {'message': u"upload failed: ./test.pdf to s3://samyakbucket/test.pdf\nHTTPSConnectionPool(host='samyakbucket.s3.amazonaws.com', port=443): Max retries exceeded with url: /test.pdf?partNumber=2&uploadId=YUYp1JYllgBQfr_lGiCzWS9XjDGncoJ38qq63JZEJWuGcyyq2rGlY5mbo3HXwmcBRc7ezkh6TEOfRmQbkfT3pkkDzT63rixYggkcNnWBb9oPK08AWVuqBDC5Ig5ePL2A (Caused by <class 'socket.error'>: [Errno 32] Broken pipe)", 'error': True}
2014-01-03 15:18:13,639 - awscli.customizations.s3.executer - DEBUG - Error calling task: Upload has been cancelled.
Traceback (most recent call last):
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/awscli/customizations/s3/executer.py", line 104, in run
    function()
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/awscli/customizations/s3/tasks.py", line 408, in __call__
    parts = self._upload_context.wait_for_parts_to_finish()
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/awscli/customizations/s3/tasks.py", line 506, in wait_for_parts_to_finish
    raise UploadCancelledError("Upload has been cancelled.")
UploadCancelledError: Upload has been cancelled.
2014-01-03 15:18:15,682 - botocore.hooks - DEBUG - Event needs-retry.s3.UploadPart: calling handler <botocore.retryhandler.RetryHandler object at 0x10129bcd0>
2014-01-03 15:18:15,683 - awscli.customizations.s3.tasks - DEBUG - Error during part upload: HTTPSConnectionPool(host='samyakbucket.s3.amazonaws.com', port=443): Max retries exceeded with url: /test.pdf?partNumber=1&uploadId=YUYp1JYllgBQfr_lGiCzWS9XjDGncoJ38qq63JZEJWuGcyyq2rGlY5mbo3HXwmcBRc7ezkh6TEOfRmQbkfT3pkkDzT63rixYggkcNnWBb9oPK08AWVuqBDC5Ig5ePL2A (Caused by <class 'socket.error'>: [Errno 32] Broken pipe)
Traceback (most recent call last):
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/awscli/customizations/s3/tasks.py", line 217, in __call__
    self._filename.service, 'UploadPart', params)
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/awscli/customizations/s3/utils.py", line 128, in operate
    http_response, response_data = operation.call(**kwargs)
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/botocore/operation.py", line 76, in call
    response = endpoint.make_request(self, params)
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/botocore/endpoint.py", line 75, in make_request
    return self._send_request(prepared_request, operation)
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/botocore/endpoint.py", line 97, in _send_request
    while self._needs_retry(attempts, operation, response, exception):
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/botocore/endpoint.py", line 128, in _needs_retry
    caught_exception=caught_exception)
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/botocore/session.py", line 554, in emit_first_non_none_response
    responses = self._events.emit(event_name, **kwargs)
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/botocore/hooks.py", line 158, in emit
    response = handler(**kwargs)
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/botocore/retryhandler.py", line 188, in __call__
    if self._checker(attempts, response, caught_exception):
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/botocore/retryhandler.py", line 255, in __call__
    caught_exception)
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/botocore/retryhandler.py", line 278, in _should_retry
    return self._checker(attempt_number, response, caught_exception)
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/botocore/retryhandler.py", line 318, in __call__
    caught_exception)
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/botocore/retryhandler.py", line 227, in __call__
    return self._check_caught_exception(attempt_number, caught_exception)
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/botocore/retryhandler.py", line 360, in _check_caught_exception
    raise caught_exception
ConnectionError: HTTPSConnectionPool(host='samyakbucket.s3.amazonaws.com', port=443): Max retries exceeded with url: /test.pdf?partNumber=1&uploadId=YUYp1JYllgBQfr_lGiCzWS9XjDGncoJ38qq63JZEJWuGcyyq2rGlY5mbo3HXwmcBRc7ezkh6TEOfRmQbkfT3pkkDzT63rixYggkcNnWBb9oPK08AWVuqBDC5Ig5ePL2A (Caused by <class 'socket.error'>: [Errno 32] Broken pipe)
2014-01-03 15:18:15,700 - awscli.customizations.s3.executer - DEBUG - Received print task: {'message': u"upload failed: ./test.pdf to s3://samyakbucket/test.pdf\nHTTPSConnectionPool(host='samyakbucket.s3.amazonaws.com', port=443): Max retries exceeded with url: /test.pdf?partNumber=1&uploadId=YUYp1JYllgBQfr_lGiCzWS9XjDGncoJ38qq63JZEJWuGcyyq2rGlY5mbo3HXwmcBRc7ezkh6TEOfRmQbkfT3pkkDzT63rixYggkcNnWBb9oPK08AWVuqBDC5Ig5ePL2A (Caused by <class 'socket.error'>: [Errno 32] Broken pipe)", 'error': True}
2014-01-03 15:18:16,702 - awscli.customizations.s3.s3handler - DEBUG - Aborting multipart upload for: test.pdf
2014-01-03 15:18:16,702 - botocore.operation - DEBUG - Operation:AbortMultipartUpload called with kwargs: {'bucket': u'samyakbucket', 'upload_id': 'YUYp1JYllgBQfr_lGiCzWS9XjDGncoJ38qq63JZEJWuGcyyq2rGlY5mbo3HXwmcBRc7ezkh6TEOfRmQbkfT3pkkDzT63rixYggkcNnWBb9oPK08AWVuqBDC5Ig5ePL2A', 'key': u'test.pdf'}
2014-01-03 15:18:16,702 - botocore.operation - DEBUG - Creating parameter objects for: Operation:AbortMultipartUpload
2014-01-03 15:18:16,703 - botocore.endpoint - DEBUG - Making request for Operation:AbortMultipartUpload (verify_ssl=True) with params: {'headers': {}, 'uri_params': {u'UploadId': 'YUYp1JYllgBQfr_lGiCzWS9XjDGncoJ38qq63JZEJWuGcyyq2rGlY5mbo3HXwmcBRc7ezkh6TEOfRmQbkfT3pkkDzT63rixYggkcNnWBb9oPK08AWVuqBDC5Ig5ePL2A', u'Bucket': u'samyakbucket', u'Key': u'test.pdf'}, 'payload': <botocore.payload.XMLPayload object at 0x101389450>}
2014-01-03 15:18:16,703 - botocore.endpoint - DEBUG - Building URI for rest endpoint.
2014-01-03 15:18:16,703 - botocore.endpoint - DEBUG - Templated URI path: /{Bucket}/{Key}
2014-01-03 15:18:16,703 - botocore.endpoint - DEBUG - Templated URI query_params: uploadId={UploadId}
2014-01-03 15:18:16,704 - botocore.endpoint - DEBUG - Rendered path: /samyakbucket/test.pdf
2014-01-03 15:18:16,704 - botocore.endpoint - DEBUG - Rendered query_params: uploadId=YUYp1JYllgBQfr_lGiCzWS9XjDGncoJ38qq63JZEJWuGcyyq2rGlY5mbo3HXwmcBRc7ezkh6TEOfRmQbkfT3pkkDzT63rixYggkcNnWBb9oPK08AWVuqBDC5Ig5ePL2A
2014-01-03 15:18:16,704 - botocore.hooks - DEBUG - Event before-auth.s3: calling handler <function fix_s3_host at 0x100fe5410>
2014-01-03 15:18:16,704 - botocore.handlers - DEBUG - Checking for DNS compatible bucket for: https://s3-us-west-1.amazonaws.com/samyakbucket/test.pdf?uploadId=YUYp1JYllgBQfr_lGiCzWS9XjDGncoJ38qq63JZEJWuGcyyq2rGlY5mbo3HXwmcBRc7ezkh6TEOfRmQbkfT3pkkDzT63rixYggkcNnWBb9oPK08AWVuqBDC5Ig5ePL2A
2014-01-03 15:18:16,704 - botocore.handlers - DEBUG - URI updated to: https://samyakbucket.s3.amazonaws.com/test.pdf?uploadId=YUYp1JYllgBQfr_lGiCzWS9XjDGncoJ38qq63JZEJWuGcyyq2rGlY5mbo3HXwmcBRc7ezkh6TEOfRmQbkfT3pkkDzT63rixYggkcNnWBb9oPK08AWVuqBDC5Ig5ePL2A
2014-01-03 15:18:16,704 - botocore.auth - DEBUG - Calculating signature using hmacv1 auth.
2014-01-03 15:18:16,705 - botocore.auth - DEBUG - HTTP request method: DELETE
2014-01-03 15:18:16,705 - botocore.auth - DEBUG - StringToSign:
DELETE


Fri, 03 Jan 2014 09:48:16 GMT
/samyakbucket/test.pdf?uploadId=YUYp1JYllgBQfr_lGiCzWS9XjDGncoJ38qq63JZEJWuGcyyq2rGlY5mbo3HXwmcBRc7ezkh6TEOfRmQbkfT3pkkDzT63rixYggkcNnWBb9oPK08AWVuqBDC5Ig5ePL2A
2014-01-03 15:18:16,705 - botocore.endpoint - DEBUG - Sending http request: <PreparedRequest [DELETE]>
2014-01-03 15:18:16,706 - botocore.vendored.requests.packages.urllib3.connectionpool - INFO - Starting new HTTPS connection (11): samyakbucket.s3.amazonaws.com
2014-01-03 15:18:18,274 - botocore.vendored.requests.packages.urllib3.connectionpool - DEBUG - "DELETE /test.pdf?uploadId=YUYp1JYllgBQfr_lGiCzWS9XjDGncoJ38qq63JZEJWuGcyyq2rGlY5mbo3HXwmcBRc7ezkh6TEOfRmQbkfT3pkkDzT63rixYggkcNnWBb9oPK08AWVuqBDC5Ig5ePL2A HTTP/1.1" 307 None
2014-01-03 15:18:18,275 - botocore.vendored.requests.packages.urllib3.connectionpool - INFO - Resetting dropped connection: samyakbucket.s3-us-west-1.amazonaws.com
2014-01-03 15:18:20,936 - botocore.vendored.requests.packages.urllib3.connectionpool - DEBUG - "DELETE /test.pdf?uploadId=YUYp1JYllgBQfr_lGiCzWS9XjDGncoJ38qq63JZEJWuGcyyq2rGlY5mbo3HXwmcBRc7ezkh6TEOfRmQbkfT3pkkDzT63rixYggkcNnWBb9oPK08AWVuqBDC5Ig5ePL2A HTTP/1.1" 204 0
2014-01-03 15:18:20,937 - botocore.response - DEBUG - Response Body:

2014-01-03 15:18:20,937 - botocore.hooks - DEBUG - Event needs-retry.s3.AbortMultipartUpload: calling handler <botocore.retryhandler.RetryHandler object at 0x10129bcd0>
2014-01-03 15:18:20,937 - botocore.retryhandler - DEBUG - No retry needed.
2014-01-03 15:18:20,937 - botocore.hooks - DEBUG - Event after-call.s3.AbortMultipartUpload: calling handler <awscli.errorhandler.ErrorHandler object at 0x1012889d0>
2014-01-03 15:18:20,937 - awscli.errorhandler - DEBUG - HTTP Response Code: 204

@samyakbhuta
Copy link

I think #454 is relevant here as well.

@1mentat
Copy link

1mentat commented Mar 7, 2014

I'm still seeing this with aws --version
aws-cli/1.3.1 Python/2.7.5 Darwin/13.1.0

upload failed: x to s3://x HTTPSConnectionPool(host='x.s3.amazonaws.com', port=443): Max retries exceeded with url: x (Caused by <class 'socket.error'>: [Errno 32] Broken pipe)

@jamesls
Copy link
Member

jamesls commented Mar 8, 2014

@1mentat Can you tell me what set of steps you're trying? Syncing? Copying? s3->local, local->s3?

Does this happen after a while, right away? One file, multiple files, etc. Anything you can share will help troubleshoot the issue.

@1mentat
Copy link

1mentat commented Mar 8, 2014

Both cp --recursive and sync seem to have the same issue. This is local to s3. It seems to mostly happen with .jars and .dlls which are in the 10s of MBs. Particularly I was uploading an unzipped version of play-1.2.7.zip from Play Framework.

@alanb1501
Copy link

I have this problem when trying to copy to a bucket in oregon. If I create it in US Standard, it works fine.

@rptb1
Copy link

rptb1 commented Apr 2, 2014

I also have this problem when creating a bucket via the Web console or via the CLI with region eu-west-1, but creating a bucket in the us-east-1 region works.

@cgwright3
Copy link

I am also have the same problem copying a 30GB zip file from an Oregon EC2 instance to Oregon S3. It does it as a multipart upload and every part gives this error.

I am use the CLI version 1.3.1.

@cgwright3
Copy link

It also works if I go to a bucket in us-east-1. In general I'd much prefer keeping everything in Oregon though.

@esammer
Copy link

esammer commented Apr 20, 2014

Getting the same error with Oregon, 98MB file, copying from local to S3, smaller files (< 1MB) work fine), version:
aws-cli/1.3.6 Python/2.7.5 Darwin/13.1.0

@jeromerg
Copy link

have the same problem... is there a workaround?

version: aws-cli/1.3.6 Python/2.7.5 Windows/7

@rpryan
Copy link

rpryan commented May 17, 2014

Same issue with 1.3 of the cli. I have a 25gb file. I am going try a different region for the bucket.

@rpryan
Copy link

rpryan commented May 18, 2014

This is reliably reproduce able in Oregon buckets and does not happen in US East.

@kevin-canadian
Copy link

I get this problem when creating NEW buckets in eu-west-1 when uploading files large enough to trigger multi-part uploading. After a few hours the uploads works fine. My problem is most likely related to #544, not sure if there is another issue as well...

@jamesls
Copy link
Member

jamesls commented Jun 9, 2014

This issue is the same as: #634, with an overview of the situation here: #634 (comment)

There's a fix for this (#634 (comment)), we just need to do more testing to ensure there are no regressions, and then we'll release the fix.

@kevin-canadian
Copy link

Thanks for the update.

@ReDrUm
Copy link

ReDrUm commented Sep 30, 2014

I was getting the same issue with 1.3.17 but having upgraded to the latest 1.4.4 it's no longer an issue.

@abellis2
Copy link

abellis2 commented Jul 6, 2018

Seems this is still an issue? I'm encountering it with large uploads:

upload failed: [...]
(32, 'EPIPE')
Completed 1717 part(s) with ... file(s) remaining

Currently on AWS CLI version:

aws-cli/1.8.7 Python/2.7.10 Darwin/16.7.0

@bchecketts
Copy link

I experienced this today, and comments above mention problems related to us-west-2 (Oregon).

I was able to upload my 500MB file to a different bucket, then copy between buckets and got it to work.

aws s3 cp /tmp/my-big-file.gz s3://working-bucket/my-big-file.gz
aws s3 cp s3://working-bucket/my-big-file.gz s3://my-desired-bucket/my-big-file.gz
aws s3 rm s3://working-bucket/my-big-file.gz

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests