Best data store for billions of rows

Posted by Jody Powlette on Stack Overflow See other posts from Stack Overflow or by Jody Powlette
Published on 2010-05-08T16:11:02Z Indexed on 2010/05/08 16:18 UTC
Read the original article Hit count: 248

Filed under:
|
|

I need to be able to store small bits of data (approximately 50-75 bytes) for billions of records (~3 billion/month for a year).

The only requirement is fast inserts and fast lookups for all records with the same GUID and the ability to access the data store from .net.

I'm a SQL server guy and I think SQL Server can do this, but with all the talk about BigTable, CouchDB, and other nosql solutions, it's sounding more and more like an alternative to a traditional RDBS may be best due to optimizations for distributed queries and scaling. I tried cassandra and the .net libraries don't currently compile or are all subject to change (along with cassandra itself).

I've looked into many nosql data stores available, but can't find one that meets my needs as a robust production-ready platform.

If you had to store 36 billion small, flat records so that they're accessible from .net, what would choose and why?

© Stack Overflow or respective owner

Related posts about sql

Related posts about server