Search Results

Search found 698 results on 28 pages for 'rsync'.

Page 10/28 | < Previous Page | 6 7 8 9 10 11 12 13 14 15 16 17  | Next Page >

• Failover tmpfs mirroring. Am I doing it right?

  - by user45286

  My goal is to have a certain directory to be available as tmpfs. There will be some modifications during server uptime in this dir and those modifications must be synced to non-tmpfs persistent dir on HDD over rsync. After server boot the latest version from non-tmpfs persistent dir must be moved to tmpfs and rsync syncing to be started. I'm afraid that rsync will erase non-tmpfs backup if tmpfs dir will be empty.. I'm doing it in this way right now: create tmpfs partition in /etc/fstab cat /etc/rc.local (pseudocode) delete "tmpfs rsync" cronjob from /var/spool/cron/crontabs if there is any cp -r /path/to/non-tmpfs-backup /path/to/tmpfs/dir append /var/spool/cron/crontabs with "tmpfs rsync" cronjob What do you think?

  Read the article

• Deploy with rsync(or svn, git, cvs) and ignore inconsistent state during deployment?

  - by zedoo

  We are currently talking about deploying a website via rsync. However, during rsyncing the application is left in an inconsistent state, as some files may already be synced while others still are left with the old version right? How do people deal with this issue? I guess the same problem exists when deploying via svn/git/cvs. Should I just close the site, rsync, and open up again? Or do people simply ignore this inconsistency problem?

  Read the article

• how to read the password from variable?

  - by Viswa

  I am trying to move my file to another system which is located in some other place, with this command: rsync -avrz src destination It works fine. But what I need is to put this command in shell script and run it like: #! /bin/sh rsync -avrz srcfilelocation destination When it runs, it asks for the destination system password. I know that password and give it manually. Now I have decided to assign the password to an environment variable, like pswd="destination system password". I need my shell script to read the password from this variable. How can I write a script to do this?

  Read the article

• Syncing large personal school-material -git-repo with things such as casual notes? Rsync, wget and Git -- or some ready tool?

  - by hhh

  My friend wants to store electrically her school -notes and process them fast, with backups. She has over 2GB -size repo already and growing all the time (mostly appended material i.e. more school notes, different formats, pdf, pictures and scanned, some text -files, etc). The goal of my friend is to process fast the notes. I suggested command like this here i.e. "# crontab -e @weekly wget --random-wait -e robots=off -U mozilla -mirror http://VeryLong.com". But I think plugging in Rsync somewhere could make it much better with Git. How would you help my friend to process and store the school -material under Git-version-controlling and still keep the size reasonable? Perhaps related rsync .git directory rsync git big repository Different scope Git/rsync mix for projects with large binaries and text files What's a good way to organize a large collection of personal scripts using git?

  Read the article

• Cygwin's RSYNC for large data transfer

  - by Tim Brigham

  I'm using rsync from Cygwin to do a large scale data transfer from an aging HP MSA 1000 to a new DAS attached to a different server. I have a daemon running on the remote server in read only mode and a local copy writing the files to disk. One of my servers is an image repository with over a million files spread across about 300 directories. Each file averages only a couple hundred kilobytes. More so than any other box this one is proving problematic. The rsync process will work for a while - some times 20 minutes, some times an hour - and then it simply quits and sits idle at a given file name. I have verified that the file isn't corrupt on the remote server and that the file is successfully created on the local drive. I ran the rsync client in -vv mode, which returns nothing. I checked out the logs created by the daemon. I looked at the network utilization on the interface, which is sitting idle. I looked at the AV settings to see if anything could pose a problem there. I even updated to the latest release of Cygwin. What do I need to in order to keep this connection up? EDIT: The client system is using the command rsync.exe server::Drives/f/Repo/ /cygdrive/T/Repo --archive -P -vv The server is using the command rsync.exe --daemon --no-detach --config "rsyncd.conf" The contents of rsyncd.conf: use chroot = false strict modes = false hosts allow = 192.168.100.9 log file = c:/rsyncd.log uid=0 gid=0 [Drives] path = /cygdrive read only = yes EDIT: The file server is 2003, the disk type on the array is GPT and the size is of the array is about 4 TB. EDIT: Stranger.. It looks like the process is reliably erroring out at about 175,000 files. Rsync runs fine when I pick the same directory it has problems with one at a time. EDIT: rsync version 3.0.9 protocol version 30 Copyright (C) 1996-2011 by Andrew Tridgell, Wayne Davison, and others. Web site: http://rsync.samba.org/ Capabilities: 64-bit files, 64-bit inums, 32-bit timestamps, 64-bit long ints, no socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace, append, ACLs, xattrs, iconv, symtimes A similar failure occurred when going from the same set of files with Cygwin to a Linux install. It didn't happen until several hours later than normal however.

  Read the article

• rsync error on Downloading

  - by Chandan

  Sir I am trying to setup a Rsyncmirror on my mount drive /mnt/Mirror/ubuntu_mirror .so I am trying the command rsync -a --bwlimit=128 rsync://archive.ubuntu.com/ubuntu /mnt/Mirror/ubuntu_mirror but it shows error rsync: getaddrinfo: archive.ubuntu.com 873: Name or service not known rsync error: error in socket IO (code 10) at clientserver.c(122) [Receiver=3.0.9] can anyone help me.. Thanks.

  Read the article

• Is it possible to rate-limit an scp/sftp/rsync/etc transfer from the command-line? ie, manual QoS on

  - by warren

  Specifically, I am looking to rate-limit an scp or sftp session (or other arbitrary network call) in the call itself. For example, let's say I want to copy 100MB to one server, and 1GB to another. I'd like to be able to run both of these at the same time, but maintain a QoS for "normal" computer usage - somewhat similar to how you can rate-limit bittorrent. Is there a way to do this without touching the networking hardware? I'm envisioning something akin to: magic-qos-tool 'scp file user@host:/path/to/file' Or.. scp -rate 40kbps file user@host:/path/to/file

  Read the article

• Is it possible to rate-limit an scp/sftp/rsync transfer from the command-line? ie, manual QoS on a s

  - by warren

  Specifically, I am looking to rate-limit an scp or sftp session in the call itself. For example, let's say I want to copy 100MB to one server, and 1GB to another. I'd like to be able to run both of these at the same time, but maintain a QoS for "normal" computer usage - somewhat similar to how you can rate-limit bittorrent. Is there a way to do this without touching the networking hardware? I'm envisioning something akin to: magic-qos-tool 'scp file user@host:/path/to/file' Or.. scp -rate 40kbps file user@host:/path/to/file

  Read the article

• rsync chown warning

  - by Ted Kim

  I try to sync two directories using rsync. the source is on Linux, and the other is on windows. So, I mount the directory on windows using the command mount -t cifs ..... in Linux system. Then I execute rsync .... Everything is OK, but rsync prints out rsync: chown "/mnt/windows/A/." failed: Permission denied (13) rsync: chown "/mnt/windows/A/readme.txt" failed: Permission denied (13) I want to sync the directories without changing ownership. How can I do? please let me know. Thanks in advance.

  Read the article

• linux + automated rsync command

  - by Diana

  my target is to copy /tmp/my_file from 10.10.10.1 to my Linux machine without login and password , I set the passwords file with the right password - secret123 so rsync should work , please advice why I get Permission denied. Remark - 10.10.10.1 address is linux machine version – red hat 5.3 rsync -WavH --password-file=/tmp/passwords --progress [email protected]:/tmp/my_file . Permission denied. rsync: connection unexpectedly closed (0 bytes read so far) rsync error: error in rsync protocol data stream (code 12) at io.c(165) more /tmp/passwords secret123 ls -ltr passwords -rwxr-xr-x 1 root root 10 Sep 12 17:32 passwords

  Read the article

• How to mirror filesystems with millions of hardlinks?

  - by Thomas Berger

  We have one big problem at the moment: We need to mirror a filesystem for one of our customers. Thats usual not really a problem, but here it is: On this filesystem there is one folder with millions of hardlinks (yes! MILLIONS!). rsync requires more then 4 days to just build the filelist. We use the following rsync options: rsync -Havz --progress serverA:/data/cms /data/ Has anyone a idea how to speed up this rsync, or use alternatives? We could not use dd as the target disk is smaller then the source.

  Read the article

• Backup files from Linux client to Windows Server

  - by Andrew

  I'm trying to backup my files from my Linux box to my Windows Server 2008 as a push, and when I delete them from my Linux box, they remain on my Windows Server. I've found lots of sources that are similar, but most results were from Windows to Linux. I managed to find slightly more similar cases like Using rsync and cygwin to Sync Files from a Linux Server to a Windows Notebook PC, and rsync from Windows PC to remote Linux server, with the most similar being a backup from Linux to Windows Server, but through a pull from the Windows Server. Initially, I used Unison because I thought having the 2-way capability would come in handy, and I would just have to set some configurations to make it 1-way. Unfortunately, I couldn't find the right configuration, and only managed to synchronize using the command unison "profile" -ui text -auto -silent. When I deleted the files on my Linux box, the files in the Server got deleted too, which of course, isn't what I want. When I tried to find any options for Unison, I only discovered the -force option, which didn't help, since what I wanted was an incremental update to the Server. I found out I could achieve this from using rsync and the -a option (archive), which would keep adding files even if I deleted them from my Linux box. I installed Cygwin on my Windows Server, configured an SSH daemon, but I can't seem to get it working. I've also already configured Windows Firewall to open port 22 (both inbound and outbound). I used the following command from my Linux box: rsync -avrzn /folder/to/be/backed/up/ [email protected]:/cygdrive/c/place/to/store/backed/up/files (a - archive, v - verbose, r - recurse into subdirectories, z - compress, n - dryrun) but it just won't work. Can anyone help me out? I don't mind using either Unison or rsync, as long as it achieves what I want.

  Read the article

• How to speed up rsync/tar of large Maildir

  - by psusi

  I have a very large Maildir I am copying to a new machine ( over 100 BaseT ) with rsync. The progress is slow. VERY SLOW. Like 1 MB/s slow. I think this is because it is a lot of small files that are being read in an order that essentially is random with respect to where the blocks are stored on disk, causing a massive seek storm. I get similar results when trying to tar the directory. Is there a way to get rsync/tar to read in disk block order, or otherwise overcome this problem?

  Read the article

• How to make NFS mounts available while offline?

  - by lpanebr

  Problem: I work on a notebook and while at work I have access to many NFS mounted drives. When I get home they are obviously not available. Windows 7 solution: My business partner uses Windows 7 and maps the folders via samba. Windows 7 has a very nice feature that let's he make these folders available offline. So when when he connects to the work network the changes get synchronized! Question: Is there a way to mimic that in ubuntu? What I have now: Server to local sync: I have added rsync entries on my crontab to copy server folders => local folders every five minutes. When at work I used the NFS mapped folders and while outside work I use the local copies. When I get at work I manually run a script that syncs local folders => server folders. Problems with my setup: slow startup when not at work (I guess do to the fstab trying to map the server folders) no conflict checking/managing I have to remember to sync manually and be careful because of the different file locations recent files do not work between work and home

  Read the article

• How does the rsync algorithm correctly identify repeating blocks?

  - by Kai

  I'm on a personal quest to learn how the rsync algorithm works. After some reading and thinking, I've come up with a situation where I think the algorithm fails. I'm trying to figure out how this is resolved in an actual implementation. Consider this example, where A is the receiver and B is the sender. A = abcde1234512345fghij B = abcde12345fghij As you can see, the only change is that 12345 has been removed. Now, to make this example interesting, let's choose a block size of 5 bytes (chars). Hashing the values on the sender's side using the weak checksum gives the following values list. abcde|12345|fghij abcde -> 495 12345 -> 255 fghij -> 520 values = [495, 255, 520] Next we check to see if any hash values differ in A. If there's a matching block we can skip to the end of that block for the next check. If there's a non-matching block then we've found a difference. I'll step through this process. Hash the first block. Does this hash exist in the values list? abcde -> 495 (yes, so skip) Hash the second block. Does this hash exist in the values list? 12345 -> 255 (yes, so skip) Hash the third block. Does this hash exist in the values list? 12345 -> 255 (yes, so skip) Hash the fourth block. Does this hash exist in the values list? fghij -> 520 (yes, so skip) No more data, we're done. Since every hash was found in the values list, we conclude that A and B are the same. Which, in my humble opinion, isn't true. It seems to me this will happen whenever there is more than one block that share the same hash. What am I missing?

  Read the article

• Quick and Dirty Backups with rsync

  It's not always the best tool for the job, but if you need to get a backup into the cloud quickly and easily, rsync might do the trick. Charlie Schluting steps you through how to build a script to do just that.

  Read the article

• Properly escaping forward slash in bash script for usage with sed

  - by user331839

  I'm trying to determine the size of the files that would be newly copied when syncing two folders by running rsync in dry mode and then summing up the sizes of the files listed in the output of rsync. Currently I'm stuck at prefixing the files by their parent folder. I found out how to prefix lines using sed and how to escape using sed, but I'm having troubles combining those two. This is how far I got: source="/my/source/folder/" target="/my/target/folder/" escaped=`echo "$source" | sed -e 's/[\/&]/\\//g'` du `rsync -ahnv $source $target | tail -n +2 | head -n -3 | sed "s/^/$escaped/"` | awk '{i+=$1} END {print i}' This is the output I get from bash -x myscript.sh + source=/my/source/folder/ + target=/my/target/folder ++ echo /my/source/folder/ ++ sed -e 's/[\/&]/\//g' + escaped=/my/source/folder/ + awk '{i+=$1} END {print i}' ++ rsync -ahnv /my/source/folder/ /my/target/folder/ ++ sed 's/^//my/source/folder//' ++ head -n -3 ++ tail -n +2 sed: -e expression #1, char 8: unknown option to `s' + du 80268 Any ideas on how to properly escape would be highly appreciated.

  Read the article

• Need an alerting system if my cloning script fails

  - by rahum

  I've configured a nightly rsync to mirror one server to a standby offsite backup server. The total datastore on the primary is 1.5TB. In the course of getting this working, I ran into numerous instabilities with the environment, which I seem to have sorted out, but even though it's now working, I am still nervous. This is intended to be a disaster-scenario standby server, and if disaster strikes and the standby does not have all the proper data synchronized, I'm out of a job. Thus, I want to script a system that will confirm, after each nightly sync, that the destination data matches the source. I realize that rsync does this, but if rsync doesn't complete fully (which was happening during the setup troubleshooting), I need to know. Any suggestions? I'm best with Ruby, if that is relevant for the solution.

  Read the article

• `rsync` NEVER uses its 'famous' delta-transfer!

  - by o_O Tync

  I have a big iso image which is currently being downloaded by a torrent client with space-reservation turned on: that means, file size is not changing while some chunks in in (4 Mib) are constantly changing because of a download. At 90% download I do the initial rsync to save time later: $ rsync -Ph DVD.iso /some/target/ sending incremental file list DVD.iso 2.60G 100% 40.23MB/s 0:01:01 (xfer#1, to-check=0/1) sent 2.60G bytes received 73 bytes 34.59M bytes/sec total size is 2.60G speedup is 1.00 Then, when the file's fully downloaded, I rsync again: total size is 2.60G speedup is 1.00 Speedup=1 says delta-transfer was not used, although 90% of the file has not changed. Why?!

  Read the article

• How do you synchronise huge sparse files (VM disk images) between machines?

  - by chrisdew

  Is there a command, such as rsync, which can synchronise huge, sparse, files from one linux server to another? It is very important that the destination file remains sparse. It may be longer (but not bigger) than the drive which contains it. Only changed blocks should be sent across the wire. I have tried rsync, but got no joy. groups.google.com/group/mailing.unix.rsync/browse_thread/thread/94f39271980513d3 If I write a programme to do this, am I just reinventing the wheel? http://www.finalcog.com/synchronise-block-devices Thanks, Chris.

  Read the article

• Only show changed files while syncing from ext4 to NTFS

  - by qox

  I would like rsync to print modified and deleted files. The verbose option (-v) does print modified files but also the list of subdirectories, maybe because touched directories are considered modified. Since I sync a lot of files from a lot of subdirectories, it's impossible to see the actual changes. So, is there a way to not print directories using rsync ? Im not looking for grep -v "*/$" kind of answers since it would also exclude new directories. Command I am using: rsync -avh --delete /media/data/src /media/data/bkp And everytime it prints the list of all directories: src/dir1/ src/dir1/sdir1/ src/dir1/sdir2/ src/dir2/ ..... Thanks for your help. EDIT: Ok, after some intensive tests .. It doesn't print all directories when syncing from an ext4 partition to an ext4 and from NTFS to NTFS. It only does when syncing from ext4 to NTFS .. And options '-c' or '--omit-dir-times' don't change that.

  Read the article

• Only show changed files with verbose option

  - by qox

  I would like rsync to print modified and deleted files. The verbose option (-v) does print modified files but also the list of subdirectories, maybe because touched directories are considered modified. Since I sync a lot of files from a lot of subdirectories, it's impossible to see the actual changes. So, is there a way to not print directories using rsync ? Im not looking for grep -v "*/$" kind of answers since it would also exclude new directories. Command I am using: rsync -avh --delete /media/data/src /media/data/bkp And everytime it prints the list of all directories: src/dir1/ src/dir1/sdir1/ src/dir1/sdir2/ src/dir2/ ..... Thanks for your help.

  Read the article

• What does the IP "192.168.1.1/24" indicate?

  - by mmattax

  I'm setting up rsync and I see this in the config file: hosts allow = 127.0.0.1 192.168.1.1/24 Does this indicate that all hosts on the local (192.168.1.*) can use rsync? It's been awhile since my networking class; I just want to understand...

  Read the article

• WHich site can provide me the FTP acces for backing UP 150GB data

  - by Mirage

  I want keep the backup of myServer data 150GB online. so that i can run rsync everyday to keep it updated. is there any reliable company which can provide the ftp/sftp access and have storage space around 200GB for onlinr backing up. I don't want to use their software to backup but my linux and rsync

  Read the article

• Is there a way to replicate a very large file shares in real-time?

  - by fsckin

  I have an hourly cron job that copies about 40GB of data from a source folder into a new folder with the hour appended on the end. When it's done, the job prunes anything older than 24 hours. This data changes very often during work hours and is on a samba file share. Here's how the folder structure looks: \server\Version.1 \server\Version.2 \server\Version.3 ... \server\Version.24 The contents of each new folder compared to the last one usually doesn't change very much, since this is a hourly job. Now you might be thinking that I'm an idiot for setting dreaming this up. Truth is, I just found out. It's actually been used for years and is so incredibly simple, anyone could delete the ENTIRE 40GB share (imagine that dialog spooling up... deleting thousands and thousands of files) and it would actually be faster to restore by moving the latest copy back to the source than it took to delete. Brilliant! Now to top this off, I need to efficiently replicate this 960GB of "mostly similar" data to a remote server over WAN link, with the replication happening as close to real-time as possible -- think hot spare, disaster recovery, etc. My first thought was rsync. Total failure. Rsync sees it sees a deletion of the folder that is 24 hours old and the addition of a new folder with 30GB of data to sync! I also looked at rdiff-backup and unison, they both appear to use similar algorithms and do not keep enough meta-data to do this intelligently. Best thing that I can find "out of the box" to do this is Windows Server "Distributed Filesystem Replication" which uses "Remote Differential Compression" -- After reading the background information on how this works, it actually looks like exactly what I need. Problem: Both servers are running Linux. D'oh! One approach to this I'm looking at is this, say it's 5AM and the cron job finishes: New Version.5 folder arrives at on local server SSH to remote server and copy Version.4 to Version.5 Run rsync on the local server pushing changes to the remote server. Rsync finally knows to do a differential copy between Version.4 and Version.5 Is there a smarter way to replicate Samba shares as close to real-time as possible? Anything out there that does "Remote Differential Compression" on Linux?

  Read the article

< Previous Page | 6 7 8 9 10 11 12 13 14 15 16 17  | Next Page >