s3cmd fails too many times
Posted
by
alfish
on Server Fault
See other posts from Server Fault
or by alfish
Published on 2011-11-12T23:41:05Z
Indexed on
2011/11/13
1:54 UTC
Read the original article
Hit count: 643
It used to be my favorite backup transport agent but now I frequently get this result from s3cmd on the very same Ubuntu server/network:
root@server:/home/backups# s3cmd put bkup.tgz s3://mybucket/
bkup.tgz -> s3://mybucket/bkup.tgz [1 of 1]
36864 of 2711541519 0% in 1s 20.95 kB/s failed
WARNING: Upload failed: /bkup.tgz ([Errno 32] Broken pipe)
WARNING: Retrying on lower speed (throttle=0.00)
WARNING: Waiting 3 sec...
bkup.tgz -> s3://mybucket/bkup.tgz [1 of 1]
36864 of 2711541519 0% in 1s 23.96 kB/s failed
WARNING: Upload failed: /bkup.tgz ([Errno 32] Broken pipe)
WARNING: Retrying on lower speed (throttle=0.01)
WARNING: Waiting 6 sec...
bkup.tgz -> s3://mybucket/bkup.tgz [1 of 1]
28672 of 2711541519 0% in 1s 18.71 kB/s failed
WARNING: Upload failed: /bkup.tgz ([Errno 32] Broken pipe)
WARNING: Retrying on lower speed (throttle=0.05)
WARNING: Waiting 9 sec...
bkup.tgz -> s3://mybucket/bkup.tgz [1 of 1]
28672 of 2711541519 0% in 1s 18.86 kB/s failed
WARNING: Upload failed: /bkup.tgz ([Errno 32] Broken pipe)
WARNING: Retrying on lower speed (throttle=0.25)
WARNING: Waiting 12 sec...
bkup.tgz -> s3://mybucket/bkup.tgz [1 of 1]
28672 of 2711541519 0% in 1s 15.79 kB/s failed
WARNING: Upload failed: /bkup.tgz ([Errno 32] Broken pipe)
WARNING: Retrying on lower speed (throttle=1.25)
WARNING: Waiting 15 sec...
bkup.tgz -> s3://mybucket/bkup.tgz [1 of 1]
12288 of 2711541519 0% in 2s 4.78 kB/s failed
ERROR: Upload of 'bkup.tgz' failed too many times. Skipping that file.
This happens even for files as small as 100MB, so I suppose it's not a size issue. It also happens when I use put with --acl-private flag (s3cmd version 1.0.1)
I appreciate if you suggest some solution or a lightweight alternative to s3cmd.
Thanks
© Server Fault or respective owner