Hi,

What I've tried:

• I have two email storage architectures. Old and new.

Old:

• courier-imapds on several (18+) 1TB-storage servers.
• If one of them show signs of running out of disk space, we migrate a few email accounts to another server.
• the servers don't have replicas. no backups either.

New:

• dovecot2 on a single huge server with 16TB (SATA) storage and a few SSDs
• we store fresh mails on the SSDs and run a doveadm purge to move mails older than a day to the SATA disks
• there is an identical server which has a max-15min-old rsync backup from the primary server
• higher-ups/management wanted to pack in as much storage as possible per server in order to minimise the cost of SSDs per server
• the rsync'ing is done because GlusterFS wasn't replicating well under that high small/random-IO.
• scaling out was expected to be done with provisioning another pair of such huge servers
• on facing disk-crunch issues like in the old architecture, manual moving of email accounts would be done.

Concerns/doubts:

• I'm not convinced with the synchronously-replicated filesystem idea works well for heavy random/small-IO. GlusterFS isn't working for us yet, I'm not sure if there's another filesystem out there for this use case. The idea was to keep identical pairs and use DNS round-robin for email delivery and IMAP/POP3 access. And if one the servers went down for whatever reasons (planned/unplanned), we'd move the IP to the other server in the pair.
• In filesystems like Lustre, I get the advantage of a single namespace whereby I do not have to worry about manually migrating accounts around and updating MAILHOME paths and other metadata/data.

Questions:

• What are the typical methods used to scale up/out with the traditional software (courier-imapd / dovecot)?
• Do traditional software that store on a locally mounted filesystem pose a roadblock to scale out with minimal "problems"? Does one have to re-write (parts of) these to work with an object-storage of some sort - such as OpenStack object storage?