Synchronize large objects to S3 efficiently
- by emk
I need to synchronize about 30GB of git repositories to S3. These repos may contain some very large pack files, on the rough order of 2GB.
I know that S3 has recently added support for large objects, and has new APIs that allow the objects to be uploaded as several parallel chunks. Is there a good command-line tool for Linux that allows me to efficiently synchronize large objects with S3 in a fashion similar to s3sync?