Search Results

Search found 698 results on 28 pages for 'rsync'.

Page 11/28 | < Previous Page | 7 8 9 10 11 12 13 14 15 16 17 18  | Next Page >

• How many simultaneous RYNC sessions can be started at once

  - by Mirage

  Just wanted to know that if Rsync is doing backup and in my cron job other rsync session times comes . Can both run simultaneously or first has to be completed before second starts

  Read the article

• Looking for suitable backup solution Mac OS X to offsite Centos 6 server 1TB of working data

  - by Brady

  I'll start by saying what we have in place currently: On site file server (Mac OS X Server) that is used by GFX designers and they have a working 1TB of data. Offsite server with 2TB available storage (Centos 6) Mac OS X server rsync data to offsite server every 6 hours (rsync -avz --delete --progress -e ssh ...) Mac OS X server does full data backup to LTO 4 tape on a 10 day recycle (Mon-Fri for 2 weeks) rsync pushes about 60GB of file changes a day. The problem: The onsite tape backup is failing as 1TB of graphics files don't compress well to fit onto a 800GB LTO4 tape. Backup is incredibly slow doing a full backup. Pain in the backside getting people to remember to change the tape. Often gets forgotten etc The quick solution: Buy LTO5 Drive and tapes. However this has been turned down because of the cost... What I would like: Something that works in the same way rysnc works. Only changed data is sent over the wire and can be scheduled to run multiple times during the day. Data that is sent is compressed and sent over SSH. Something that keeps a 14day retention but doesn't keep duplicate data So as an example if I have 1TB of working data and 60GB of changes are made each day then I expect around 1.84TB of data to be stored on the offsite server. To work with the Mac OS X server and Centos 6 server. Not cost an arm and a leg. Must be a cheaper solution than buying an LTO5 drive with tapes (around £1500). Be able to be setup to run autonomously. Have some sort of control panel that will allow an admin to easily restore a file/folder. Any recommendations?

  Read the article

• How to sync Ubuntu/software/configurations between N computers with free software and/or without a cloud?

  - by skanatek

  Note: this question is not about syncing data in a Dropbox-like way (files, folders), it is more about syncing configurations. I would like to have exactly the same version of Ubuntu with all the software installed and configured both on my Desktop PC and on my Laptop PC (and maybe on my small netbook PC) without using Ubuntu Sync and with minimal maintenance effort (setup once, run for a long time). The use case is the following: I work on my Laptop PC and do some changes to software configuration, for example: configure vim to have a new plugin update the Search Tracker / Recoll file search index configure Thunderbird to have an additional IMAP account ('remember password') add some new bookmarks in Firefox/Chrome change the desktop background image install new software with apt-get install build and install new software with checkinstall etc. I do some 'sync' operation I switch to my Desktop PC and get all the changes from (1) working on the Desktop PC I work on my Desktop PC and do some changes to software configuration, for example: add new directory to the list of directories to be backed up by DejaDup add a new check spelling dictionary to the Libreoffice Writer configure the Terminator software to have colored fonts install new font into the Ubuntu system configure Ekiga to make phone calls etc. I do some 'sync' operation I switch to my Laptop PC and get all the changes from (1) and (4) working on the Laptop PC. Question: What free/open-source software can I use to sync both machines' Ubuntu systems, installed software and configurations? Is it possible to do that without any cloud services? Complementary question: It is obvious that the Desktop PC and the Laptop PC have different hardware configurations. How does the 'sync software' in question deal with video drivers, wlan drivers and their configurations? Note: I do not need all the PCs to be synced at the same time, because I work with only one single machine at once. Note: I considered to use Chef to solve the problem, but it seems that it might be really cumbersome to maintain such a setup. Note: I also considered using a bootable USB with Ubuntu installed (portable Linux), but I am not sure that the video drivers will work then.

  Read the article

• Something like Dropbox for local use

  - by Casper

  I am looking for a solution to sync folder pairs between a NAS and multiple local macs. Each of the macs could edit files and the other macs should then get synced automatically. Basically my own local version of Dropbox without using "cloud-storage". I have looked into solutions using rsync. As I understand it rsync is not really capable of doing a bi-directional sync. I also do not want to necessarily invoke the sync process. I would prefer a daemon running in the background - waiting and checking for changes and then syncing them "live". The program should also be flexible enough to recognize that it sometimes (in the case with laptops) can not reach the NAS. It should then just wait for the connection to be back again, without bugging me ever few minutes. I have looked into synk, folderwatch, rsync and a few others, but I haven't really found a solution. Isn't there something like "offline folders" from microsoft for the mac? Thanks PS: just for clarification - I don't want to sync for backup purposes, instead I am wanting to sync so that all macs have a local copy of the most recent changes to files.

  Read the article

• rsyncing - ignoring checksums

  - by paulj3000

  Trying to copy a bunch of data over and time, unfortunately, is of the essence. I'd like to do an rsync of all the data in one direction, pretty much rsync just clobbers what's over on the destination server. Is there a way to do an rsync and just say, "overwrite all files" Is there a better way of doing this? We're talking 500GB of data that only has to go in one direction.

  Read the article

• s3fs Input/output error

  - by shadow_of__soul

  i'm trying to set up a backup system with s3fs and the amazon s3 service. i followed this 2 guides: http://qugstart.com/blog/linux/how-to-mount-an-amazon-s3-bucket-as-virtual-drive-on-centos-5-2/ http://blog.eberly.org/2008/10/27/how-i-automated-my-backups-to-amazon-s3-using-rsync/ anyway making a tail to the /var/log/messages i get: Aug 28 13:37:46 server s3fs:###response=403 i already tried creating the authentication file on /etc/passwd-s3fs and setting there the access and private key, passing it trough the command line, i checked several times the credentials and i used it with s3fox, and is working. i also have set the time of the machine (with the date command) to be the same as the amazon S3 servers (i got the time of the S3 server uploading a file with the file manager) not only rsync don't work, commands like ls or cp in the /mnt/s3 didn't work also. any help of how i can solve/debug this? Regards, Shadow.

  Read the article

• Immutable hard links on ext3/4?

  - by shovas

  In my research on file versioning at the fs level, snapshotting, and related ideas, I took a look at hard-links and exactly what they are and how they behave. Using rsync you can get a pretty slick poor man's snapshotting system up and running on file systems that don't natively support it. But, can you get immutable hard links on ext3/4 or any other file systems for that matter? My definition for immutable hard link is: A hard link which, when changed on one location, becomes a regular copy and no longer a hard link. I would like this because it would enable snapshotting use of the source data to link against instead of a copy of the data (in the case of the rsync snapshotting technique). I have gigabytes of data that can't be duplicated due to space restrictions but I have enough room if I can intelligently snapshot individual changed files with the rest linked to the source not a copy. Given all that, is there some other technique, feature or technology I'm really looking for?

  Read the article

• Best way to backup and restore millions of files

  - by bongo

  Hi, I'm facing a rebuilding of the volume on which I host the mail storage (kerio mailserver, which uses maildirs). I need to backup and restore as quick as possible the 3.5+ millions (for about 600GB) small files of the store directory. It takes more than 12 hours via rsync to a NFS share, but I also have a 1TB firewire 800 raid1 disk that I can use (from some preliminary tests it's faster). I'm working off a XServe intel. What is the fastest way to do it? Rsync? Finder copy? tar?

  Read the article

• Is there a remote file transfer command that preserves nanosecond timestamps?

  - by Denver Gingerich

  I've tried transferring files using scp and rsync on Ubuntu 10.04, but neither of them preserves more than second precision. Here's an example: $ touch test1 $ scp -p test1 localhost:test2 $ ls -l --full-time test* -rw-r--r-- 1 user user 0 2011-01-14 18:46:06.579717282 -0500 test1 -rw-r--r-- 1 user user 0 2011-01-14 18:46:06.000000000 -0500 test2 $ cp -p test1 test2 $ ls -l --full-time test* -rw-r--r-- 1 user user 0 2011-01-14 18:46:06.579717282 -0500 test1 -rw-r--r-- 1 user user 0 2011-01-14 18:46:06.579717282 -0500 test2 $ A straight copy works fine, but scp truncates the timestamp. Are there any tools (preferably similar to scp or rsync in their usage) that do remote file transfers while preserving nanosecond timestamps? I could write a hacky script to do it, but I'd rather not.

  Read the article

• How to manage iowait over cifs?

  - by Silvia

  For backup purposes we have Cifs file Server running that contains encrypted containers for backing up the more sensitive data. The container is mounted with cryptsetup and loop as a local filesystem and the rsync is used for backups. Because the Cifs server is not the fastest machine ever built, running the rsync process results in an iowait on the servers running the backup which in turn drives Nagios into an email frenzy. The question is, how do reduce the iowait on the server? Configuring Nagios to not report seems more like a workaround then a solution. Stretching the backups over different time intervals is already done with little effect and spending money is also not an option because apparently, we are talking about a "non-critical system".

  Read the article

• Tool or script to detect moved or renamed files on Linux prior to a backup

  - by Pharaun

  Basically I am searching to see if there exists a tool or script that can detect moved or renamed files so that I can get a list of renamed/moved files and apply the same operation on the other end of the network to conserve on bandwidth. Basically disk storage is cheap but bandwidth isn't, and the problem is that the files often will be reorganized or moved around into a better directory structure thus when you use rsync to do the backup, rsync won't notice that its a renamed or moved file and re-transmission it over the network all over again despite having the same file on the other end. So I am wondering if there exists a script or tool that can record where all the files are and their names, then just prior to a backup, it would rescan and detect moved or renamed files, then I can take that list and re-apply the move/rename operation on the other side. Here's a list of the "general" features of the files: Large unchanging files They can be renamed or moved around [Edit:] These all are good answers, and what I end up doing in the end was looking at all of the answers and will be writing some code to deal with this. Basically what I am thinking/working on now is: Using something like AIDE for the "initial" scan and enable me to keep checksums on the files because they are supposed to never change, so it would aid on detecting corruption. Creating an inotify daemon that would monitor these files/directory and recording any changes relating to renames & moving the files around to a log file. There are some edge cases where inotify might fail to record that something happened to the file system, thus there is a final step of using find to search the file system for files that has a change time latter than the last backup. This has several benefits: Checksums/etc from AIDE to be able to check/make sure that some media did not get corrupt Inotify keeps resource usage low and no need to re-scan the filesystem over and over No need to patch rsync; If I have to patch things I can, but I would prefer to avoid patching things to keep the burden lower, (IE don't need to re-patch everytime there is an update). I've used Unison before and its really nice, however I could've sworn that Unison does keep copies around on the filesystem and that its "archive" files can grow to be rather large?

  Read the article

• Elastix, how to MOVE files from one server to other server?

  - by yudayyy

  In my office, i have to schedule for moving a file from one computer to other computer (Both are using Elastix). My idea is using cron, scp, and rm to do this. So here are the script that i use: scp -r /home/data/* [email protected]:/home/data1 && rm -r /home/data/* That script did the copy, but not remove the source file. I already read this question: Hov to _MOVE_ files with scp? The problem is, the computer doesn't have an internet connection. So i cannot install rsync on my elastix computer. yum install rsync Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile and then it freezes. Any idea how to do this?

  Read the article

• Cloning OpenVZ container

  - by Tiffany Walker

  I have an OpenVZ container on 1 host and I would like to clone it over to my server. both run SolusVM. I only have root access to my server and would like to host the container on my server now. Can I use rsync to clone the drive while the OS is running on both? Using a command like this: rsync -uazPx --exclude='/boot' --exclude='/proc' --exclude='/dev' --exclude='/lib' --exclude='/tmp' --exclude='/var/lock' / [email protected]:/ Is there any other areas I should probably not copy over?

  Read the article

• Backup of images

  - by Sam Kong

  I've just installed a Ubuntu for a file server. It will share a folder (samba) and employees of my company will save photos on that. Currently the total amount of the photos is about 100GB and every day 20MB will be added. My question is about backup plan. I want to backup the photos to a remote server using a cron job. I can think of 2 things. rsync git Image files won't be changed so rsync will do. But as people say, I must git all my data. What would you do? Thanks. Sam

  Read the article

• How should I organize my backups ?

  - by Patrick

  I'm using for the first time rsync to create daily backups of my websites and I was wondering if I should overwrite the previous copy or should I create multiple copies and overwrite only the oldest one ? (I might not have enough space for that, though). I actually have also this question. Let's suppose most of files are accidentally erases.. does rsync delete all these files from the backup space because they don't exist anymore ? How does exactly work in this case ? thanks

  Read the article

• Backing up an rsnapshot directory to a remote device

  - by user123480

  I have a local backup server that uses rsnapshot with hard links that contains about 10TBs of information which add about 4 to 5GBs per day. It's works great. I've been requested to set up and maintain a remote backup of the local rsnapshot directory structure. It's a nightly backup. I've tried using rsync with encryption which takes forever and eats system resources. A previous post says not to use rsync with hard links for that reason. I need a suggestion of how I can keep the local and remote copies of the rsnapshot structures in sync? Thanks

  Read the article

• Options to efficiently synchronize 1 million files with remote servers?

  - by Zilvinas

  At a company I work for we have such a thing called "playlists" which are small files ~100-300 bytes each. There's about a million of them. About 100,000 of them get changed every hour. These playlists need to be uploaded to 10 other remote servers on different continents every hour and it needs to happen quick in under 2 mins ideally. It's very important that files that are deleted on the master are also deleted on all the replicas. We currently use Linux for our infrastructure. I was thinking about trying rsync with the -W option to copy whole files without comparing contents. I haven't tried it yet but maybe people who have more experience with rsync could tell me if it's a viable option? What other options are worth considering?

  Read the article

• How to set/keep directory permissions?

  - by Dylan

  I'm using CwRsync to connect from my Windows development machine to a linux webserver : rsync -avuz -e ./ssh --exclude=".svn" /cygdrive/c/xampp/htdocs/project123/ [email protected]:/home/user123/public_html This syncs my development project directory nicely and fast to the server. But after doing this, all directory properties are reset to the local user user123 only, so the website is not available anymore. I need to manually reset those properties. Why is this happening, and how to prevent it? PS. coming from a Windows environment I'm having a really hard time understanding rsync. I copied the above command from some examples... just need to get this one small thing working too...

  Read the article

• Reliable file copy (move) process - mostly Unix/Linux

  - by mfinni

  Short story : We have a need for a rock-solid reliable file mover process. We have source directories that are often being written to that we need to move files from. The files come in pairs - a big binary, and a small XML index. We get a CTL file that defines these file bundles. There is a process that operates on the files once they are in the destination directory; that gets rid of them when it's done. Would rsync do the best job, or do we need to get more complex? Long story as follows : We have multiple sources to pull from : one set of directories are on a Windows machine (that does have Cygwin and an SSH daemon), and a whole pile of directories are on a set of SFTP servers (Most of these are also Windows.) Our destinations are a list of directories on AIX servers. We used to use a very reliable Perl script on the Windows/Cygwin machine when it was our only source. However, we're working on getting rid of that machine, and there are other sources now, the SFTP servers, that we cannot presently run our own scripts on. For security reasons, we can't run the copy jobs on our AIX servers - they have no access to the source servers. We currently have a homegrown Java program on a Linux machine that uses SFTP to pull from the various new SFTP source directories, copies to a local tmp directory, verifies that everything is present, then copies that to the AIX machines, and then deletes the files from the source. However, we're finding any number of bugs or poorly-handled error checking. None of us are Java experts, so fixing/improving this may be difficult. Concerns for us are: With a remote source (SFTP), will rsync leave alone any file still being written? Some of these files are large. From reading the docs, it seems like rysnc will be very good about not removing the source until the destination is reliably written. Does anyone have experience confirming or disproving this? Additional info We will be concerned about the ingestion process that operates on the files once they are in the destination directory. We don't want it operating on files while we are in the process of copying them; it waits until the small XML index file is present. Our current copy job are supposed to copy the XML file last. Sometimes the network has problems, sometimes the SFTP source servers crap out on us. Sometimes we typo the config files and a destination directory doesn't exist. We never want to lose a file due to this sort of error. We need good logs If you were presented with this, would you just script up some rsync? Or would you build or buy a tool, and if so, what would it be (or what technologies would it use?) I (and others on my team) are decent with Perl.

  Read the article

• Syncronize Linux /etc/ directory

  - by entend

  I have virtual machine with Linux (Ubuntu server) which is used as prototype for other machines. Sometimes I make changes in prototype system and want to import this changes at some other machine. I know about Puppet, cfengine and FAI but want something easy for example rsync script which will work through ssh when it needed. Main goal is /etc/ directory. But I don't want to syncronize some private files for example /etc/passwd /etc/shadow and so on. I don't know all of it. Are there tips for my task ? May be someone have such rsync script.

  Read the article

• Transfer using linux ssh and maintaining permissions

  - by jbolt

  I need to transfer files across ssh to another server. The file structures are identical on both sides. I have used scp -r but that does not retain the orginal file/dir permissions. rsync does the job of keeping the permissions in tact but does not delete the files on the destination side if I want to overwrite them because of changes. I know rsync will write the changes when the source files are newer but I need it to just copy everything reguardless of the date (ie replace destination directory with the one I am moving) without having to shell into the destination first and manually delete the dir. I heard tar can do this but I can not seem to get it to work without errors. The syntax is tar -cf - /directory/directory | ssh host.name tar -xf - C /destination_directory Any help would be appreciated.

  Read the article

• How can I rename files and subdirectories in a copied directory based on changes in the original?

  - by GaryF

  I have a directory structure with many hundreds of files and folders underneath it for organising files (in this case photos). I create backups of that directory structure by rsyncing it to identical copies on an external drives periodically. These drives may be offsite some of the time. I want to restructure and rename the files and directories in the original and then, later, when I have an external drive onsite, be able to run some tool that will cause these structural and naming changes to happen on the backup. If I just us rsync, it'll have to recopy much of the data to the backup drive, which I'd rather avoid due to the sizes involved. How can I get the changes I make to the original directory into the backups, as they become available, without having to recopy/rsync the data?

  Read the article

• How can make rsync use sudo

  - by Bryan M.

  I use rsync to mirror a number of folders on our failover server. However, some of our files, such as thumbnails or full-text indexes, are generated by our applications under the web user (named 'nobody'), and default to restrictive permissions. Also, I'm doing this over ssh, where root access is disabled, and I'd like to keep it that way, if possible. Is there any reasonable way I can tell rsync to run as sudo? Or should I look into changing the file permissions? Thanks

  Read the article

• rsync directories

  - by Buzzzz

  Hello, I'm trying to sync my music collection between my mac and my linux workstation but fails to mirror from my server share to linux. rsync -avz --progress Music/ /volumes/myserver/music works fine but the reverse on my linux ws doesnt. rsync -avz --progress /path/to/samba/share/music/ ~/Music does nothing.. Any clue what I have gotten wrong with this?? Best Regards Anders Olme

  Read the article

• How do I improve my incremental-backup performance?

  - by Alistair Bell

  I'm currently using the traditional rsync+cp -al method to create incremental/snapshot backups of our server tree. The backups are going onto a pair of eight-disk towers connected to the backup machine (a Sandy Bridge machine with 16 GB of RAM, running CentOS 5.5) via four eSATA connections (four disks per connection). Each disk is a regular 2 TB disk, so we have 32 TB of disk space connected to the backup machine. We're backing up about 20 TB of data on the servers with this. The problem is that each daily backup is taking more than 24 hours, and the real time-killer isn't the actual rsync, but the time it takes to perform a cp -al of the tree locally on the backup machine. It's taking more than 12 hours just to make the shadow copy of the tree, and as far as I can tell the performance backlog is at the disk (top shows the cp using a lot of RAM but not a lot of CPU and mostly in uninterruptible-sleep state) We have the server data split into four major volumes (and a few minor ones), and each of these backups runs in parallel (with some offsets in the cron to try to get some disks' cp done first). There are two volumes on the backup drive, both striped LVM volumes of 16 TB each. So obviously I need to improve the performance because it's unusable as it stands. The first question is: when CentOS 6 comes out, with support for btrfs, will making snapshots of subvolumes with btrfs substantially increase this performance? The second is: is there a way, with ext3 or something else supported in CentOS 5 or 6, to 'encourage' it to put the directories/inodes in one part of a volume (which could happen to be the part that's on an SSD, via LVM) and the files in another? That would presumably solve the problem, but I don't know of ways to hint ext3 like that.

  Read the article

< Previous Page | 7 8 9 10 11 12 13 14 15 16 17 18  | Next Page >