Running an rsync sweep before initializing lsyncd for synchronizing instances on EC2
Posted
by
chrisallenlane
on Server Fault
See other posts from Server Fault
or by chrisallenlane
Published on 2011-11-16T22:15:15Z
Indexed on
2012/03/31
17:32 UTC
Read the original article
Hit count: 275
amazon-ec2
My company uses several EC2 servers that will scale up and down according to the load we're receiving on our sites at any given moment. For the sake of our discussion here, we're running four instances:
- master.ourdomain.com - the file syncing "hub" of the webservers
- www1/www2/www3.ourdomain.com - three webservers which turn on or off as dictated by load
I'm using lsyncd to keep all of the webservers in sync, and for the most part, it's working quite well. We're using a two-way syncing scheme, such that each webserver syncs against master, and master syncs against each webserver. Thus, the webservers are kept in sync, even though they aren't syncing against each other directly.
I'm having one problem that I'm having a hard time solving,though. It occurs under these circumstances:
- When changes are made on master (perhaps after we've pushed new code), while some of the redundant webservers are sleeping
- And then a sleeping webserver wakes-up to absorb load
Under that circumstance, I would like the following to happen:
- First, the newly-awoken webserver should sync its file structure - one way - against master, to bring its web application code up-to-date.
- Then, and only then, should it begin pushing changes in its file structure back to master.
Unfortunately, currently, when a sleeping server is started, when lsyncd starts up, it pushes changes back to master before updating its own codebase, thus overwriting new code with old.
Thus, before lsyncd starts, I'd like to be able to synchronize the webservers code against master's, perhaps by running a simple one-way rsync against the two machines.
We're running lsyncd v.2, and I've tried to make this happen by using the "bash" configuration options documented in the lsyncd manual. My configuration file looks like this:
settings = {
logfile = "/home/user/log/lsyncd/log.txt",
statusFile = "/home/user/log/lsyncd/status.txt",
maxProcesses = 2,
nodaemon = false,
}
bash = {
onStartup = "rsync [email protected]:/home/user/www /home/user/www"
}
sync{
default.rsyncssh,
source="/home/user/www/",
host="[email protected]",
targetdir="/home/user/www/",
rsyncOpts="-ltus",
excludeFrom="/home/user/conf/lsyncd/exclude"
}
(I've obviously redacted that file somewhat to protect the identities of the guilty.)
Simply put, though, this just isn't working.
How else might I approach this problem? I was looking at the --delete-after
option in man rsync
, but I don't think that does what I'm looking for.
Are there any suggestions about how I should approach this problem?
Thanks for lending your time and expertise.
Chris
© Server Fault or respective owner