How long does it take in practice to warm up large in-memory databases?

Posted by Sim on Server Fault See other posts from Server Fault or by Sim
Published on 2012-10-21T00:29:41Z Indexed on 2012/10/21 11:04 UTC
Read the original article Hit count: 166

Filed under:
|
|
|

Companies such as Peak Hosting are offering 64 core machines with 512Gb RAM for $2K/month. This is a very interesting choice for in-memory databases such as Memcached/Redis as well as databases whose performance degrades rapidly when the data & indexes don't fit in RAM, such as MongoDB.

My main concern with monster machines such as these is the time it takes to warm up an in-memory database. In my experience, theoretical metrics, e.g., that SATA can load 100Mb/sec, fall short of what happens in practice. Even at that rate, 100Mb/sec means that loading up 512Gb RAM machine from SATA disks can take over 1 1/2 hours (!).

I am looking for real-world reports of warm-up times for machines with very large memory. Please, share details of the software on the machine, data size, storage configuration, e.g., SATA or SSD, network, hosting/cloud provider, if relevant, etc.

© Server Fault or respective owner

Related posts about mongodb

Related posts about caching