Best Practice: Migrating Email Boxes (maildir format)

Posted by GruffTech on Server Fault See other posts from Server Fault or by GruffTech
Published on 2011-01-09T09:31:09Z Indexed on 2011/01/09 9:55 UTC
Read the original article Hit count: 237

Filed under:

linux

|

email

|

maildir

So here's the situation.

I've got about 20,000 maildir email accounts chewing up a several hundred GB of space on our email server. Maildir by nature keeps thousands of tiny a** little files, instead of one .mbox file or the like...

So i need to migrate all of these several millions of files from one server to the other, for both space and life-cycle reasons.

the conventional methods i would use all work just fine. rsync is the option that comes immediately to mind, however i wanted to see if there are any other "better" options out there.

Rsync not handling multi-threaded transfers in this situation sucks because it never actually gets up to speed and saturates my network connection, because of this the transfer from one server to another will take hours beyond hours, when it shouldn't really take more then one or two.

I know this is highly opinionated and subjective and will therefore be marked community wiki.

© Server Fault or respective owner

Related posts about linux

apt-get install and update fail

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
I've got a problem with apt-get update and apt-get install ... commands . every time update or installing fails and errors are : Get:1 http://dl.google.com stable Release.gpg [198B] Ign http://dl.google.com/linux/chrome/deb/ stable/main Translation-en_US Get:2 http://dl… >>> More
kernel module compiling error

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
sh@ubuntu:/home/ccpp/helloworld$ make gcc-4.6 -O2 -DMODULE -D_KERNEL_ -W -Wall -Wstrict-prototypes -Wmissing-prototypes -isystem /lib/modules/`uname -r`/build/include -c -o hello-1.o hello-1.c hello-1.c:4:0: warning: "MODULE" redefined [enabled by default] <command-line>:0:0: note: this is… >>> More
Build-Essentials installation failing

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
I am having trouble accessing the several critical header files that show to be a part of the build process. The "Ubuntu Software Center" shows "Build Essentials" as installed: Next I did the following two commands, which did not improve the problem: ~$ sudo apt-get install build-essential [sudo]… >>> More
Updating Debian kernel

as seen on Super User - Search for 'Super User'
I'm trying to update my Debian machine to 2.6.32-46 (which is the new stable). However, after doing apt-get update my apt-cache search linux-image shows me: linux-headers-2.6.32-5-486 - Header files for Linux 2.6.32-5-486 linux-headers-2.6.32-5-686-bigmem - Header files for Linux 2.6.32-5-686-bigmem linux-headers-2… >>> More
Serial connection over a single USB cable (Windows to linux, or linux to linux)

as seen on Server Fault - Search for 'Server Fault'
I'm helping out with a project for an embedded device that only has USB and no serial. This device is running Linux. These days, when we need to connect to a serial port on a device we typically use a USB to serial adapter (on something like a phone system or a load balancing device, etc). I would… >>> More

Related posts about email

CRM 2011 - Workflows Vs JavaScripts

as seen on Programmers - Search for 'Programmers'
In the Contact entity, I have the following attributes Preferred email - A read only field of type Email Personal email 1 - An email field Personal email 2 - An email field Work email 1 - An email field Work email 2 - An email field School email - An email field Other email - An email field Preferred… >>> More
Sending Email in Android using JavaMail API without using the default android app(Builtin Email app

as seen on Stack Overflow - Search for 'Stack Overflow'
Hi Folks, I am trying to create a mail sending application in android, If I use `Intent emailIntent = new Intent(android.content.Intent.ACTION_SEND`); This will launch the builtin application of android, I'm trying to send the mail on button click directly without using this app, Please any suggestions… >>> More
Processing Email in Outlook

as seen on Daniel Moth - Search for 'Daniel Moth'
A. Why Goal 1 = Help others: Have at most a 24-hour response turnaround to internal (from colleague) emails, typically achieving same day response. Goal 2 = Help projects: Not to implicitly pass/miss an opportunity to have impact on electronic discussions around any project on the radar. Not achieving… >>> More
Processing Email in Outlook

as seen on Daniel Moth - Search for 'Daniel Moth'
A. Why Goal 1 = Help others: Have at most a 24-hour response turnaround to internal (from colleague) emails, typically achieving same day response. Goal 2 = Help projects: Not to implicitly pass/miss an opportunity to have impact on electronic discussions around any project on the radar. Not achieving… >>> More
email is moved to junk-email folder even though sender is listed as safe

as seen on Super User - Search for 'Super User'
Using Outlook-2007 : Added a sender as safe sender, but still the email from that id goes to junk-email folder. Tried resetting all settings and restarted outlook . any ideas ? Thanks! >>> More