Zimbra to .PST

[seachnasaigh]

We've worked out nearly all the bugs with migrating to Zimbra, and we're about ready to call Sales and get some quotes to stand the live puppy up.

One outstanding issue: how do we migrate off retired users to a standardised file format that we can burn to a CD.

In the past, we've stripped off users' Exchange emails to a .pst file, burned the file to a disk, and handed it to the appropriate department head for whatever they want to do with it (it's our server, it's their data ... that model).

I've searched the Z forums quite a lot for this, and this seems to be lacking. I've done some fruitless googling on it also. Clearly, Z has a way to handle "turning off" a user in a quite elegant way. But ... how do I get that user off of my licence list, and how do I archive off that user's email to a file that is usable by a non-IT department head?

My first thought was that there MUST be some equivalent reverse procedure of .pst > Zimbra but, alas, so far, nothing shows up. I've seen the post in a couple of places, but the posts are old. Anything new here? Doesn't have to be .pst, just something I can make useful for the suits.

Thanks.

--ckr

[Baylink]

I'm pretty sure you can *find* a way to do this; the internal storage format is, very roughly, Maildir -- there's a big directory full of RFC 2822 text files, pointed to by the message story database.

So there should be *some* way to make a directory tree with all the user's messages and burn said tree to a disc or make a tarball.

I just don't know that it's in there *yet*.

[seachnasaigh]

Och aye. I've parsed through the backend of Postfix and Zimbra's implementation thereof. That's not the point.

I could craft a script that grabs that and turns it into a tarball that I could burn down. However, I have to assume that a) the person that takes over my job when I get hit by the bus doesn't have my chops and b) that over time, the linux distro and the suite itself change to the point that my brilliantly written script tanks for some obscure library/path/kernel issue and that the person who takes my job when I get hit by the bus doesn't understand how to fix it, either.

When I'm looking at supported software, in any form, I have to assume a LCD for a sysadmin that has an A+/Net+ cert and basic Linux admin capabilities. I can't assume that my enterprise always hires a clone of me (sooper geiniuses are hard to come by) and that anything I leave behind after I assume room temperature has to be manageable by someone who didn't invent it.

That's the point I'm trying to make: Zimbra has done a wonderful job thus far of making a collaboration suite that is designed by geniuses for use by morons. There's a piece missing, I think ... I'm asking if they've addressed that, and if not, I'm suggesting oh-so-subtly that they should. The Zimbra Outlook .pst > Zimbra connector is brilliant; we need one that goes the other way (or at least goes to some generic RFC compliant package) so that a) we have a way to conveniently EOL some user's mailbox and b) the latter-day saints out there can do it without deep linux knowledge, much as the rest of the suite is configured.

By the bye, (and offtopic) we seem to share a love of Heinlein and Glenmorangie. Cheers to you, mate.

--ckr

[Baylink]

Well, after "och, aye", I will change my estimation of the name, in another thread, from Irish Gaelic to Scots Gaelic, and I'm sure I won't know how to pronounce it. :-)

I don't disagree with you; if you look at the pile of bugs with my name on them, I suspect you'll conclude we have the same opinion; that of Jerry Pournelle about Vulcan (dBase I):

"Infuriatingly excellent."

That is, Z gets so much stuff right that it's worth being annoyed about the little stuff, because there's not that much big stuff to worry about.

It sounds like you, even more than me, are in a position to write the code of which you speak and contribute it -- remember, Z is (arguably) an open source project, albeit one with a ridiculously steep intake-hill to climb.

I do wish we could inveigle some of the development team in here for more than just answers to questions...

Anyway, nice to find someone else with a brain and literate English floating around the forums. :-)

[seachnasaigh]

As I mentioned in the other thread, sorry to throw you; NI is famous for having Scots phrases mixed in with Irish.

Thanks for the props, but I'm always concerned these days about the longevity and overall portability of any script I write. In olden times, I was cowboy enough to perl/regex my way out of any bad situation. Today, with many more years and enterprises under my belt, and as a SysAdmin/Manager hybrid (beJayzus, I feel like a hyphenated hyundai) I tend to look at the code as something that has a life, and a lifespan. It has to have a paladin, someone to maintain it over time, or it's just a band-aid. I don't like to see ad-hoc scripts in production software anymore. If you're going to write a script for open source, that's fine if you're willing to own and maintain it. If you're writing it for something that is (even arguably) open source but also corporate-supported, then someone in the corporation has to adopt it, bless it, and own it into production model, or again, it's just another bloody band-aid. If you understand what I'm getting at, I'm leery of being the author of that band-aid, however useful it may be at the time. It's not just solving a problem; it's creating a solution, and that latter involves more than just writing the original code to solve the problem.

That said, and working within the framework provided by Zimbra, we do have a solution to this (and we're testing this week to see if it holds up). The solution is to keep one licence of Outlook sitting around (hell, we've paid for it, we own it into perpetuity) and when we need to archive off a user, we'll use Zimbra's native Outlook connector to attach to that user's mailbox. Then, we can convert that down to a .pst that we can archive and, if need be, attach to another user's mailbox in future. This is a little cumbersome, but not too much so and is similar to what we do now (Exchange > .pst; .pst >> Exchange data file).

Here's the business case that this is designed for: Our written procedure is that when an employee leaves the enterprise, his/er mail is burned down to a disk and given to a department head. If the individual IS a department head, it's given to the VP/CFO/CIO/CM level manager. The model: we own the sandbox, management owns the sand. What you do with a departed employee's email isn't our job; it's just our responsibility to hand it to you. That mail/contact/calendar etc info can be used by management, given to the next employee to fill that slot as reference, whatever. We can import it back into the system as need be at the time the new employee is hired or give the manager the ability to view/integrate that into their workflow.

Zimbra is very good stuff. We (IT that is) like it a LOT. But the EOL for an employee, and the sum of what they've contributed/communicated for the enterprise, is NOT trivial to management. We (and Z, as a collaborative suite) need to be sensitive to that. How to transition that information is not trivial to a business enterprise; employees are often transient, but the value they contribute, and their communications as a part of that, are not. Any collaborative business suite needs to be sensitive to that. Zimbra is great on the bringing-in end, but some attention needs to be paid to the going-out end as well.

I agree with you; we should have developers read this. We're the link between them and the suits who run the enterprises that ultimately pay for all this.

Thanks for the engaging discourse. I hope we can build a better beast with it.

Cheers.

--ckr

[Baylink]

I understand and agree with your model, and it's one of many things I think they need to be visibly paying more attention to.

Love the sandbox metaphor. :-)

And the beast ain't too damned bad, either.

[bdial]

couldnt' you just use the outlook connector? it brings down all the mail from the server into a pst.

[Dirk]

Here's a suggestion for a slightly 'off the wall' method.

Given that the goal is to allow access to the ex-staff mailbox by certain current staff for the purposes of reference; I'd shy away from burning the data to a CD as it may not fit. Ok, DVD; well it may still not fit and Bluray burners are expensive, and people will still lose the disk.

So what have we left?

My idea, that's what. Build a linux server running VMware server (free). Install multiple VM's onto that server, each running linux, install Zimbra desktop onto each of them. Fire up a VM when a staff leaves and sync the account to it, name it for that staff and then lock the VM so changes are not saved (take snapshot and revert to snapshot on poweroff)

This would allow secure storage of multiple accounts, that could be accessed easily from any computer so long as suitable access rights are given.

Done. Done and done.

[Baylink]

Elegant, but a bit frangible.

What happens if/when Zimbra makes enough changes that that Desktop won't talk to the server anymore?

No, they really need to come up with a way to import and export mailstores independent of mailboxes.

I'm going to start a design thread for this, and when we lock down exactly what we want, I'll RFE it.

[mmorse]

Another solution (ignoring my employee hat for a moment that would love people buying lots of NE licenses when they need to keep users around in perpetuity). We haven't yet decided on a 'left the company' kinda account status that wouldn't account against licenses, but could keep everything intact for re-activation. Some would go one step further and argue that when 'inactive' shares should still be intact so other co-workers could continue to collaborate on anything needed, and just block new mail and logins - but I think that definitely borders on 'usage of the account'...)

PST's are one way I suppose, but that assumes their MS oriented...
You could do backups, which used to not be as cross-version capable, but see some new improvements: Recent Admin Tidbits - Part 1 » Zimbra :: Blog (Pretend for a moment on that one I'm talking about just archiving purposes & not 'handoff' to mgt.)
Imapsync is just mail so you'd have to export other stuff using additional methods. User Migration - Zimbra :: Wiki
zimbraMigrate is handy and incorporates Imapsync + REST SourceForge.net: ZCS Tools

What happens if/when Zimbra makes enough changes that that Desktop won't talk to the server anymore?

Just browsing that works fine - you don't need to upgrade as it's not syncing to a live account. I assume you mean if you want to connect the manager account in it to transfer; well it'll be a 2nd account in ZD, and you can't copy between them at the moment anyways. (Some desktop RFE's open on this.)
Dirk's solution about VM's is great, you could even take that one step further really and apply it to a FOSS box that restores to snapshot unless you turn off the revert function to add another user.

The approach: A 2nd FOSS box (or re-import when needed)

You can export an entire mailbox from the NE box with:
zmmailbox -z -m user@domain.com getRestURL "//?fmt=tgz" > /tmp/user.tgz

And import on the FOSS box (or back on the NE when needed) with:
zmmailbox -z -m user@domain.com postRestURL "//?fmt=tgz&resolve=reset" /tmp/user.tgz

(obviously shares aren't a concern at this point)

The resolve= parameter has several options:
-"skip" ignores duplicates of old items (the default resolution).
-"modify" changes old items.
-"reset" will delete the old subfolder (or entire mailbox if /).
-"replace" will delete and re-enthem. (Reset will be a bit faster on an empty destination mailbox because it skips most dup checks.)

Don't remember when we added .tgz, works reliably fresh in 5.0.9 (with some additional duplication fixes in 5.0.10) - but going back far into the past there's the .zip formatter:
http:// server.domain.com/user/~/?fmt=zip&query=is:anywhere

(might prove interesting on 10G mailboxes - I haven't tried)

Or even use a zimlet to do it:
http://www.zimbra.com/forums/adminis...end-users.html (.eml)
http://www.zimbra.com/forums/zimlets...-txt-file.html (.txt or html) < Browse 'em straight off the DVD

You have so many methods for putting messages back into Zimbra; CURL/zmmailbox addMessge/zmlmtpinject etc:
Recover data from store folders
Moving Folders between users

Heck even outlook express does .msg & .eml
Import EML and MSG email message files into Outlook

And yes there's a few RFE's already:
Bug 19630 - Migration Tool zimbra to zimbra
Bug 29573 - Enhance 'zmmailboxmove' to move mailboxes between different ZCS version
Bug 30163 - zimbra-zimbra migration via tar formatter