[SOLVED] Migrate Entourage + POP

[supersport]

Everyone,
We have zcs-NETWORK-5.0.8_GA_2462.UBUNTU6. We have a variety of users installed, but we've just now moved into support Mac users.

Long term, these users will be using the web client, so I'm not worried about usability on that end...it's just the migration that's the bear.

For calendar and contacts, we plan to use the iSync connector, but the problem lies in their emails. They previously POP-ed their email into Entourage. We want all their emails on our Zimbra server.

I'm at a loss about how to efficiently get the email from Entourage to Zimbra.

So far, we've been doing the following:

1. Creating an IMAP account to Zimbra
2. Copying the emails from the POP account to the IMAP account
3. Letting the IMAP sync the email to Zimbra

This actually works pretty well, until we came across some users who have a lot of folders. This just makes the process very tedious and makes that users system unusable while we do the migration (because we admins have to take it over for long periods of time).

So, at this point, I see that we have three potential solutions:

1. Continue to use the IMAP method, and slog through each folder one at a time
2. Discover that Entourage uses some sort of .pst file, and we can import that somehow
3. Discover that iSync can indeed import email in addition to cal and contacts

Please, for the love of Pete, tell me that options 2 or 3 above are possible. Help.

I debated weather or not to post this here, or to the iSync forum. Since I don't think that the iSync connector is appropriate for migrating actual emails, this forum won out. If I'm wrong, and iSync connector can be used to move emails from Entourage to the zimbra server, please let me know! I would love to be wrong on that account.

Thanks in advance for any thoughts or assistance you guys can offer.

<addition/edit>I did search the forums/wiki, and the closest thing I could find to address this issue was this post, but it is specifically not focused on email migration, just cal/contacts</addition/edit>

[supersport]

Update: I found this site (external) that seems to indicate I can easily export the Entourage POP emails to mbox format. I don't yet know if this is a step forward or not.

If I am able to export the mbox files, and then load them to a server, could I use some utility to import them? It looks like from the wiki that I can use a script to import the emails. The only question/concern with this, then is will this method maintain the original recipt datestamp? Thisis an important component to the migration.

Thanks in advance for any thoughs you may have...

[supersport]

It looks like we got this all wrapped up.

For posterity, here are the steps we took to do the migration:

1. (From the client Mac) In Entourage, click on every folder and drag it to the desktop. This includes sub folders and such. Each time you do this, you will create an .mbox file.
2. (optional) If you have a lot of folders, and especially sub folders, now might be a good time to take note of the folder hierarchy.
3. Upload the .mbox files to the zimbra server. Since Mac's are now BSD based, we just fired up a Terminal and scp-ed them over. You may also choose a thumb drive or other method, depending on your setup.
4. (From the zimbra server) Install mb2md. It is available in rpm and other forms. On our system, we were able to install it via:

   apt-get install mb2md

5. (optional, but highly recommended) Make a backup copy of the mbox files now.
6. Now, you want to clean up the .mbox files. This requires several steps/caveats. It also helps if you understand a little about the mbox and maildir formats, but I'll try to be clear.
   1. (optional, but highly recommended) Change the filenames of all files/directories to not contain spaces. You can go back later and change this back, but it makes life much easier when you're doing things to not have to worry about escaping spaces.
   2. We had a problem with the formatting of the .mbox files. It seems that it was a similar issue to the whole DOS vs Unix carriage return deal. To fix it, we ran the following:

      perl -e 's/^M/\n/g' -p -i `find /root/mboxmail -type f`

      Please note that we had all of our .mbox files in /root/mboxmail, and that the "^M" character was created via "CTRL+V CTRL+M". If you just type the keys ^M, the above won't work.
   3. Now, here's the tricky part (for me at least). A mailbox folder can contain both email, and other folders. That doesn't seem earth shattering, but it makes the migration a little trickier. What you need to do is separate your mbox files into two categories: 1) files that don't have sub-folders, and 2) files that have subfolders within them. Let's address the first category first. You essentially want to create a directory structure that mirrors your folder structure. That is, if you had the following folder structure:
      

      Personal
      Personal/Family
      Personal/Friends
      Business/
      Business/Projects
      Business/Marketing

      Then, you want to have your mbox files look like:

      /root/mboxmail/Personal/Family.mbox
      /root/mboxmail/Personal/Friends.mbox
      /root/mboxmail/Business/Projects.mbox
      /root/mboxmail/Business/Marketing.mbox

   4. Now, remove the .mbox extension from the files. The resulting file structure will look like:

      /root/mboxmail/Personal/Family
      /root/mboxmail/Personal/Friends
      /root/mboxmail/Business/Projects
      /root/mboxmail/Business/Marketing

      (The lack of the extension to differentiate between a directory and a mailbox file is why you have to handle the files two different ways.)
   5. You will now do a similar setup for the 'folders-with-email'. If you only have one level of nested folders, this isn't too bad, but you may have to do this several times if you have multiple levels of folders. Also note that this will need to be in a separate directory structure:

      /root/mboxmailroots/Personal.mbox
      /root/mboxmailroots/Business.mbox

   6. Finally, remove the .mbox extensions...

      /root/mboxmailroots/Personal
      /root/mboxmailroots/Business

7. Run the initial mb2md, for the regular folders (from /root):

   mb2md -s mboxmail -R -d /tmp/mboxmail

8. And for the 'folders-with-email' (again, from /root):

   mb2md -s mboxmailroots -R -d /tmp/mboxmail

   NOTE, you may need to do this step multiple times depending on your level of 'folder nesting'.
   Once this is done, you'll have all of the mbox emails in maildir format in /tmp/mboxmail
9. The last few steps are done as zimbra, so let's prepare the maildir files (and become zimbra):

   chown -R zimbra:zimbra /tmp/mboxmail
   su - zimbra

10. Create the folders on the account. If you only have one or two folders, you're best served by automating this.
    1. Continuing our example from above, create a file (/root/folders.txt) with the contents:

       createFolder /Personal
       createFolder /Personal/Family 
       createFolder /Personal/Friends
       createFolder /Business
       createFolder /Business/Projects
       createFolder /Business/Marketing

    2. Then, run the following command (replace "<user>" with the username/email of the account you are importing to):

       zmmailbox -z -m <user> < /root/folders.txt

11. Finally, import the email into the zimbra folders.
    1. Again, create a file (/root/import.txt) with the contents:

       addMessage Personal /tmp/mboxmail/.Personal/cur
       addMessage Personal/Family /tmp/mboxmail/.Personal.Family/cur
       addMessage Personal/Friends /tmp/mboxmail/.Personal.Friends/cur
       addMessage Business /tmp/mboxmail/.Business/cur
       addMessage Business/Projects /tmp/mboxmail/.Business.Projects/cur
       addMessage Business/Marketing /tmp/mboxmail/.Business.Marketing/cur

    2. Finally, run the following command (replace "<user>" with the username/email of the account you are importing to):

       zmmailbox -z -m <user> < /root/import.txt

12. (optional) Clean up any folders you renamed to remove spaces. I did this by logging into the web client for that user and renaming the appropriate folders. I'm sure there's an easier/better way, but I know that works.

Please forgive the excessive length of this post, but this was a complicated migration and I want to make sure that others learn from my trial-and-error. I was able to put most of this together from various posts around the forums, but I didn't find anyone who detailed how tie everything together. I truly hope that this can be useful to others.

Stu