Skip to content

Script to copy an AWS S3 Bucket to another bucket using many threads in Python

Notifications You must be signed in to change notification settings

paultuckey/s3_bucket_to_bucket_copy_py

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

25 Commits
 
 
 
 
 
 
 
 

Repository files navigation

s3_bucket_to_bucket_copy.py

Script to copy all files (or files with given prefix) from an AWS S3 Bucket in one region to another bucket in a different region using many threads in Python.

Add a file, .s3cfg, to your home directory with the following format (or download s3cmd and run "s3cmd --configure"):

[default]
access_key = <your access key here>
secret_key = <your secret key here>

Then run:

s3_bucket_to_bucket_copy.py <source_bucket_name>[/<prefix>] <dest_bucket_name> 

Note, For OSX users, make sure the Python boto package is installed:

sudo easy_install boto
pip install -U boto

When dealing with a very large number of files, you might want to try to use more worker threads:

s3_bucket_to_bucket_copy.py <source_bucket_name>[/<prefix>] <dest_bucket_name> -t 100

Developed by us guys at Showcase Workshop.

Other Contributors:

About

Script to copy an AWS S3 Bucket to another bucket using many threads in Python

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Contributors 4

  •  
  •  
  •  
  •  

Languages