How to copy a 200GB file faster?

Posted by RainDoctor on Server Fault See other posts from Server Fault or by RainDoctor
Published on 2010-03-31T20:28:37Z Indexed on 2010/03/31 20:33 UTC
Read the original article Hit count: 363

Filed under:

scp

|

copy

|

rsync

|

copying

I got a 200GB .tgz file on server A(RHEL 5.2). I wanna transfer that file to server B (RHEL 5.3). Server B is on ESXi 4 Update1. I gave 10GB to that Server B VM, with 4 vCPUs.

Both Server A and Server B are connected with an ethernet cable with local IP addies (no switch involved)

scp gives me about 3Mbps. Is there a way to get 400Mbps?

© Server Fault or respective owner

Related posts about scp

Variable directory names over SCP

as seen on Server Fault - Search for 'Server Fault'
We have a backup routine that previously ran from one disk to another on the same server, but have recently moved the source data to a remote server and are trying to replicate the job via scp. We need to run the script on the target server, and we've set up key-based scp (no username/password… >>> More
Does SCP lock the file it is transfering?

as seen on Server Fault - Search for 'Server Fault'
Have a situation where we have an application log sitting out on an AIX server. The log is being continually written to from the application, and we have users on Windows who want to view the file. What they've been doing is using WinSCP to transfer the file to their desktop and they open it using… >>> More
Problems with SCP stalling during file copy over VPN

as seen on Server Fault - Search for 'Server Fault'
I have a series of files I need to copy via SCP over a VPN to a remote linux server each night. The files are not large, we're talking about tens of megabytes here, but the file copy almost always stalls after a few seconds. Running the SCP command with -vvv, I see the following over and over throughout… >>> More
scp hangs at 100%, before finishing

as seen on Super User - Search for 'Super User'
Hi, my scp commands do hang at 100% before the finish (some seconds), The output I get using -vvv is this: http://paste.pocoo.org/show/137504/ Any ideas? >>> More
Prevent scp from copying local files?

as seen on Super User - Search for 'Super User'
I just read the Linux scp command issue question and it reminded me that I regularily forget to specify the colon in the host part of a scp command, and thus copying a file locally instead of copying to a remote host, e.g. I do scp foo host instead of scp foo host: But I never use scp to copy… >>> More

Related posts about copy

encfs error while decoding the data

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
I have installed encfs and started using it to secure all my personal & office data and it was working absolutely fine until 2 hours back. The setup is like this. I have a folder in Copy folder called OfficeData which gets synchronized with my Copy folder When I login into the system I use… >>> More
XSL How to copy identical + add copy with attributes changes

as seen on Stack Overflow - Search for 'Stack Overflow'
Hello I need to copy a node and its sub-nodes: first: one identical copy followed by a modified copy with some attributes values changed Here is the extract to change: <Configuration Name="Debug|Win32" OutputDirectory=".\Debug" IntermediateDirectory=".\Debug" ATLMinimizesCRunTimeLibraryUsage="FALSE" … >>> More
Problems with wireless file copy - hangs during copy

as seen on Super User - Search for 'Super User'
I have this problem on my home network where when I try to copy files via wireless connection, at some point the copy fails and the machine doing the copy loses connection to the network. I'm pretty much lost as to figuring out the source for the problem... Some more details that might help: It's… >>> More
Copy files in Linux, avoid the copy if files do exist in destination

as seen on Super User - Search for 'Super User'
Hi, I need to copy a /home/user folder from one hardisk to another one. It has 100000 files and around 10G size. I use cp -r /origin /destination sometines I get some errors due to broken links, permissions and so on. So I fix the error, and need to start again the copy. I wonder how could… >>> More
AMD 24 core server memory bandwidth

as seen on Server Fault - Search for 'Server Fault'
I need some help to determine whether the memory bandwidth I'm seeing under Linux on my server is normal or not. Here's the server spec: HP ProLiant DL165 G7 2x AMD Opteron 6164 HE 12-Core 40 GB RAM (10 x 4GB DDR1333) Debian 6.0 Using mbw on this server I get the following numbers: foo1:~# mbw… >>> More