What is fastest way to backup a disk image over LAN?

Posted by David Balažic on Super User See other posts from Super User or by David Balažic
Published on 2014-08-23T12:50:14Z Indexed on 2014/08/23 16:25 UTC
Read the original article Hit count: 234

Filed under:
|
|
|
|

Sometimes I boot sysrescd or a similar live linux on a PC to backup the hardrive over local network to my server. I noticed many times, that the transfer speed is not optimal (slower than HDD and network speed).

Any rules of thumb what to do and what to avoid?

What I typically do is something like:

dd bs=16M if=/dev/sda | nc ... # on client
nc ... | dd bs=16M of=/destination/disk/backup1 # on server

I also "throw" in lzop (other are way too slow) and sometimes on the fly md5sum calculation (both of uncompressed and compress source). I try to add (m)buffer (or other alternatives) to improve throughput (and get a progress indicator).

I noticed that even with enough free CPU, adding commands to the pipeline slows things down.

Typically the destination is on a NTFS volume (accessed via ntfs-3g, with the _big_writes_ option).

© Super User or respective owner

Related posts about linux

Related posts about networking