Homebrew large data cluster access for 2 user levels?
Posted
by Yegor
on Server Fault
See other posts from Server Fault
or by Yegor
Published on 2010-03-30T05:37:48Z
Indexed on
2010/03/30
5:43 UTC
Read the original article
Hit count: 313
The title probably makes little sense, so here is an example.
I have a file hosting site, that serves a large amount of semi-randomly accessed files.
The setup is as follows:
- High horsepower front-end +DB server that also does encoding for files that need encoding
- Fresh file server, which stores newly uploaded content, thats probably (and usually) rapidly accessible, which has 500GB of raided SSD storage, that can push over 3GBit of traffic.
- 3 cheap node servers, containing 2 x 750GB SATA drives in raid1, where files older than 2 weeks are archived, from the SSD server (mentioned above).
Files on each server are accessed via subdomains (via modsec) in a straight forward fashion (server1.domain.com, server2.domain.com, etc)
Where I have the problem is this. I introduced a "premium" service where people pay a small fee every month, and get ad-free, quick accesses to stuff on the site. Once they are logged in, they access same files via premium.server1.domain.com via a different modsec script, with a different pass phrase. That all works fine and dandy.... except the cheap node servers are all IO bound, so accessing the files on them via a different, unsaturated network makes no difference, since it cannot read off the drive fast enough.
What would be a good way to make files on the site be accessible via 2 different network routes, 1 of which will be saturated (the "free network") while all other files are on an un-saturated "premium" network?
© Server Fault or respective owner