How to rsync a large file, with as little CPU and bandwidth expense as possible?

Posted by Johan Allgoth on Super User See other posts from Super User or by Johan Allgoth
Published on 2010-06-16T11:46:28Z Indexed on 2010/06/16 11:53 UTC
Read the original article Hit count: 278

Filed under:

rsync

|

economics

I have a 500 GB file that I plan on backing up remotely. The file changes often. I'll be rsyncing it from a desktop to a server. Both can run rsync client or server.

What is the proper command for this? The ones I've tried sofar has been taking forever or simply acted strange.

Example and results:

rsync -cv --partial --inplace --no-whole-file /desktop/file1 myserver.com::module/file1

Seems to work, but only if I do it twice (?!). Also, slow.

Does the above command do the checksumming on both computers, or only on the sending one? Is it correct otherwise?

© Super User or respective owner

Related posts about rsync

Rsync: how to mount truecrypt on-the-fly on the receiving side?

as seen on Super User - Search for 'Super User'
The short version: how can I keep an rsync backup on a truecrypt volume? The hard part is to mount/unmount this volume on the fly when it is needed for rsync. Details This is my current backup configuration (which works fairly well for the most part): backup source is on Win7 64 bit, destination… >>> More
Two Questions on for Rsync - rsync by date and by file name

as seen on Stack Overflow - Search for 'Stack Overflow'
Hi, I have two questions with respect to rsync: 1: I have a bunch of files which are incremented by day of the year. Ex: file.txt.81, file.txt.82, etc. Now, these files are in different directories: data1/file.txt.81 data1/file.txt.82 data2/file2.txt.81 data2/file2.txt.82 How can I have rsync… >>> More
Cygwin rsync broken

as seen on Super User - Search for 'Super User'
I get an error with cygwin rsync trying to transfer files between local - remote host. Any ideas? C:\>rsync user@host:~/file newfile Password: rsync: connection unexpectedly closed (0 bytes received so far) [sender] rsync error: error in rsync protocol data stream (code 12) at io.c(601) [sender= 3… >>> More
macport selfupdate not working

as seen on Super User - Search for 'Super User'
macbookpro:~ eistrati$ port -v MacPorts 2.1.2 macbookpro:~ eistrati$ xcodebuild -version Xcode 4.5.2 Build version 4G2008a macbookpro:~ eistrati$ sudo port -d selfupdate DEBUG: Copying /Users/eistrati/Library/Preferences/com.apple.dt.Xcode.plist to /opt/local/var/macports/home/Library/Preferences DEBUG:… >>> More
Rsync from godaddy to OS X

as seen on Super User - Search for 'Super User'
I would like to use rsync to backup my website to my local computer (OS X). I started of with this guide and got pretty far. I use the following rsync-line: rsync -PzrlptgD --del --delete-excluded -r --rsync-path=~/bin/rsync user@server:~/ /local/backup/folder/ I wanted to use the -a option (same… >>> More

Related posts about economics

Should economics be taught to geeks?

as seen on Stack Overflow - Search for 'Stack Overflow'
When I interview developers for jobs I'm really astonished by their lack of economics consciousness. It seems they think they'll be recruited just for technological fun. When you ask them for technical solutions they would just follow hypes from software editors without consideration for the economics… >>> More
Good Economics book for developers

as seen on Stack Overflow - Search for 'Stack Overflow'
Joel mentions in several of his blog posts that it is very important for a developer/software entrepreneur to have solid understanding of Economics. Yet the Fog Creek MBA book reading list does not include any Economics books. Is there any good material that people can recommend? Obviously, I am… >>> More
Enterprise Cloud Computing: Risk and Economics

as seen on Internet.com - Search for 'Internet.com'
Cloud computing can help optimize a company's capital investments by reducing its costs for hardware, software and real estate, resulting in a much lower total cost of ownership and, ultimately, a whole new way of looking at the economics of operational IT. >>> More
Enterprise Cloud Computing: Risk and Economics

as seen on Internet.com - Search for 'Internet.com'
Cloud is here, it's real and it's rapidly maturing to the point where the benefits are starting to appeal to those looking to solve enterprise risk and economic issues. >>> More
Enterprise Cloud Computing: Risk and Economics

as seen on Internet.com - Search for 'Internet.com'
Cloud is here, it's real and it's rapidly maturing to the point where the benefits are starting to appeal to those looking to solve enterprise risk and economic issues. >>> More