Best data store for billions of rows
Posted
by Jody Powlette
on Stack Overflow
See other posts from Stack Overflow
or by Jody Powlette
Published on 2010-05-08T16:11:02Z
Indexed on
2010/05/08
16:18 UTC
Read the original article
Hit count: 248
I need to be able to store small bits of data (approximately 50-75 bytes) for billions of records (~3 billion/month for a year).
The only requirement is fast inserts and fast lookups for all records with the same GUID and the ability to access the data store from .net.
I'm a SQL server guy and I think SQL Server can do this, but with all the talk about BigTable, CouchDB, and other nosql solutions, it's sounding more and more like an alternative to a traditional RDBS may be best due to optimizations for distributed queries and scaling. I tried cassandra and the .net libraries don't currently compile or are all subject to change (along with cassandra itself).
I've looked into many nosql data stores available, but can't find one that meets my needs as a robust production-ready platform.
If you had to store 36 billion small, flat records so that they're accessible from .net, what would choose and why?
© Stack Overflow or respective owner