Hi,
Is there a key value store that will give me the following:
Allow me to simply add and remove nodes and will redstribute the data automatically
Allow me to remove nodes and still have 2 extra data nodes to provide redundancy
Allow me to store text or images up to 1GB in size
Can store small size data up to 100TB of data
Fast (so will allow queries to be performed on top of it)
Make all this transparent to the client
Works on Ubuntu/FreeBSD or Mac
Free or open source
I basically want something I can use a "single", and not have to worry about having memcached, a db, and several storage components so yes, I do want a database "silver bullet" you could say.
Thanks
Zubair
Answers so far:
MogileFS on top of BackBlaze - As far as I can see this is just a filesystem, and after some research it only seems to be appropriate for large image files
Tokyo Tyrant - Needs lightcloud. This doesn't auto scale as you add new nodes. I did look into this and it seems it is very fast for queries which fit onto a single node though
Riak - This is one I am looking into myself, but I don't have any results yet
Amazon S3 - Is anyone using this as their sole persistance layer in production? From what I have seen it seems to be used for storage of images as complex queries are too expensive
@shaman suggested Cassandra - definitely one I am looking into
So far it seems that there is no database or key value store that fulfills the criteria I mentioned, not even after offering a bounty of 100 points did the question get answered!