I'm working on an application that requires a large amount of storage space and I want to handle storage 'in-house' (Much cheaper than, say, S3) so we will have multiple servers (Initially 4) with large amounts of storage (6TB each).

The storage will need to be very flexible and configurable, each piece of data should be replicated on at least 2 servers and must be easily readable/writable from ether an API of a UNIX device/file/folder like a normal drive, I don't mind which. We must also be able to easily offload content to our HTTP CDN (Edgecast), it doesn't need to have built in HTTP support but if it doesn't I'm going to have to write something to get the files onto HTTP so they can be pulled by the CDN.

I've looked at a lot of solutions including

• Eucalyptus Walrus
• OpenStack Object Storage
• MogileFS
• and some others which I can't remember

All the servers will be running RHEL 6, they have 4x1.5TB drives which will be RAID1'd into a single partition. All the servers have 1GB/s connections between them and 100MB/s connections to the internet with unlimited bandwidth. They have 2x2.66ghz processors.

I understand there isn't a single, perfect answer but it would be nice to get some pointers.