I've just bought a 6-core Phenom with 16G of RAM. I use it primarily for compiling and video encoding (and occassional web/db). I'm finding all activities get disk-bound and I just can't keep all 6 cores fed. I'm buying an SSD raid to sit between the HDD and tmpfs.

I want to setup a "layered" filesystem where reads are cached on tmpfs but writes safely go through to the SSD. I want files (or blocks) that haven't been read lately on the SSD to then be written back to a HDD using a compressed FS or block layer.

So basically reads: - Check tmpfs - Check SSD - Check HD

And writes: - Straight to SSD (for safety), then tmpfs (for speed)

And periodically, or when space gets low: - Move least frequently accessed files down one layer.

I've seen a few projects of interest. CacheFS, cachefsd, bcache seem pretty close but I'm having trouble determining which are practical. bcache seems a little risky (early adoption), cachefs seems tied to specific network filesystems.

There are "union" projects unionfs and aufs that let you mount filesystems over each other (USB device over a DVD usually) but both are distributed as a patch and I get the impression this sort of "transparent" mounting was going to become a kernel feature rather than a FS.

I know the kernel has a built-in disk cache but it doesn't seem to work well with compiling. I see a 20x speed improvement when I move my source files to tmpfs. I think it's because the standard buffers are dedicated to a specific process and compiling creates and destroys thousands of processes during a build (just guessing there). It looks like I really want those files precached.

I've read tmpfs can use virtual memory. In that case is it practical to create a giant tmpfs with swap on the SSD?

I don't need to boot off the resulting layered filesystem. I can load grub, kernel and initrd from elsewhere if needed.

So that's the background. The question has several components I guess:

• Recommended FS and/or block layer for the SSD and compressed HDD.
• Recommended mkfs parameters (block size, options etc...)
• Recommended cache/mount technology to bind the layers transparently
• Required mount parameters
• Required kernel options / patches, etc..