Large Enterprise Environments.

[bigpulla]

Hi there,

I have been looking at replacing our current mailserver farm with zimbra but have a few concerns.

I was wondering if there are any other administrators out there using zimbra in a large enterprise environment that could offer me some advice in regards to how you combated the following problems i have.

* It seems Zimbra can only have one active mailstore node in a cluster. Therefore Zimbra appears to be not easily scalable.
* Can Zimbra mailboxes be managed directly (i.e. Copy/paste/delete mails).
* It looks like Zimbra's own mailbox stores and backup procedures means that we cannot take full advantage of the NetApp snapshot feature.
* Zimbras own backup procedures also means that we now need two large file systems to accommodate both live & backed up emails... Plus we still have to put it to tape!
* If i ever have to restore a couple of mails from a point in time i would I have to restore the whole mailbox?

Can anyone suggest any answers or work arounds in regards to these problems or is Zimbra just not meant for large environments???

[bdial]

or this

650,000 Mailbox Migration Qmail-LDAP 2 Zimbra NE

[bigpulla]

Thats great! Once again an answer that doesnt tell me anything? How are they are they looking to setup zimbra? Surely not with a single active/standby node. Surely they are going to hit alot of the problems i described above.

I have contacted sales 3 times now and only once recieved a reply (which was a question about my question). Much like your reply yesterday (which im still waiting for some feedback on?).

Im sorry if i seem annoyed but i have wasted more than enough time trying to find information on zimbra that would suggest its is capable of such a large deployment. And i would be more than interested to know how comcast intend to support more than 24 million customers using zimbra. Your glamorous media release doesnt really touch on that????

Are they going to have over a minutes worth of downtime while the cluster brings down the failed node and starts the standby's services?

[bdial]

i'll attempt to answer hte ones i have any experience in (i'ven ever done a cluster install)

Originally Posted by bigpulla

* Can Zimbra mailboxes be managed directly (i.e. Copy/paste/delete mails).

you mean as an admin? there is no global mailbox manager utility, but an admin can from the admin interface select an account and then start a session as that user so they're looking at their mailboxes. There are also several handy command line utils

* It looks like Zimbra's own mailbox stores and backup procedures means that we cannot take full advantage of the NetApp snapshot feature.

you could use it for disaster recovery but in general snapshotting a filesystem with databases and indexes in use isn't a good idea. you normally have to stop any active locks on tables and what not. this is why zimbra puts accounts in maintenance mode briefly while it backs up parts of them.

* Zimbras own backup procedures also means that we now need two large file systems to accommodate both live & backed up emails... Plus we still have to put it to tape!

yeah depending on how you're goingn to do backups you could potentially need as much space as your mailstore is in order to do a full backup of all accounts. Of course you can compress with the -z option for zmbackup but you can't be sure of what results you'll get. There is that a "grouped backup" mode that may be better for large installations you can read about it in the docs. for long term archival its up to you wether you're doing tape or some sort of data deduplicaiton device

* If i ever have to restore a couple of mails from a point in time i would I have to restore the whole mailbox?

yeah if you need to restore messages of an account you need to restore the entire account to the target date/time as another name and then copy the messages back

Can anyone suggest any answers or work arounds in regards to these problems or is Zimbra just not meant for large environments???

that press release about comcast was a bit old, they have implimented it (i'm using it right now) for however many millions of accounts they have. and there are several posts underneath the success stories forums of installations in the tens and hundres of thousands. I understand you're frustrated but your question isn't if zimbra is meant for large environments. That much has been proven already. Your question is if zimbra is meant for YOUR environment. That is really the specialty of zimbra's sales engineers. I don't work for zimbra so I can't help you there.

[Klug]

Originally Posted by bigpulla

* It seems Zimbra can only have one active mailstore node in a cluster. Therefore Zimbra appears to be not easily scalable.

ZCS is highly scalable, horizontaly scalable.
Getting more users? Add a mailbox server...

You can cluster (HA) the mailbox servers (and other servers too) if you wish.

Originally Posted by bigpulla

* Can Zimbra mailboxes be managed directly (i.e. Copy/paste/delete mails).

I don't understand the question.
The users can manage their mailboxes (ZWC or heavy client).
The admins have the ability to connect to a user mailbox through admin webUI.

Originally Posted by bigpulla

* It looks like Zimbra's own mailbox stores and backup procedures means that we cannot take full advantage of the NetApp snapshot feature.

Snapshot _is_not_ backup.

Originally Posted by bigpulla

* Zimbras own backup procedures also means that we now need two large file systems to accommodate both live & backed up emails... Plus we still have to put it to tape!

Yes.
However, not sure you need to put it on tape as ZCS' backup integrates rotation...

Originally Posted by bigpulla

* If i ever have to restore a couple of mails from a point in time i would I have to restore the whole mailbox?

Yes.
Then copy/move the messages from the restored mailbox to live mailbox.

Originally Posted by bigpulla

Can anyone suggest any answers or work arounds in regards to these problems or is Zimbra just not meant for large environments???

I'm not sure your questions are related to "using ZCS in large environments", I think they're related to "can ZCS fits my habits in the way I work"...

[Rich Graves]

Yes, there is a one-to-one correspondence between Zimbra mailbox servers and /opt/zimbra volumes. It sounds like you're coming from a background where multiple POP heads (with some IMAP support tacked on) all accessed an NFS-mounted spool. Zimbra does indexing and caching way beyond the point where that could work. You can scale Zimbra horizontally with an arbitrary number of mailbox servers. Only 2-node active/passive clusters are documented and supported, requiring 2N servers for HA, but it's not terribly difficult to build an N+1 cluster if you know your way around RedHat or Veritas cluster suites.

I've found SAN snapshots useful with Zimbra for per-message restores and for testing. Mount a read/write snapshot of /opt/zimbra on a VM, boot it up, copy just the messages you need. Always stage upgrades on a snapshot before touching the production environment.

Zimbra backups de-duplicate. Keeping 30 days on disk (the default) doesn't require much more storage than a week. What is your email retention policy?

Use cost-correct storage. I have my /opt/zimbra on a midrange SAN, and /opt/zimbra/backup on a low-end FC-to-SATA tray several kilometers away.

[bigpulla]

Hi guys, thanks for some good replys. Rich i like the idea of mounting the snapshots. But how do you copy the emails from one server to another?

At the moment we are currently using the qmail toaster (dont laugh, believe me i know the pro's and con's). But apart from the fact that its a dog to setup and it doesnt have the collaboration, log managment and webUI features of zimbra. The acctual administration of mailboxes is a pleasure. At the moment i have a farm of servers all accessing the same store. These servers are lightning fast and are all load balanced. They can be brought up and down without users even noticing anything has happened. If i ever need to expand i can just clone one of the VM's bring it up and its away. But i have had to build all my own user interfaces for managing alot of the features zimbra users would take for granted and the UI that i built does not even compare to zimbra.

Klug - I realise snapshots is not a backup, give me a little credit. At the moment i can use snapshots to retrive any recently deleted emails and copy them straight into the users mailboxes without having to access the tapes or having to resort to any long winded exercises in getting them back. My biggest concern was if i had to restore to a certain point in time, would I loose all mail recieved after that restore point? And even though that answer seems to be answered it still looks like it will be a pain. I cant tell you the amount of times a user has told me i acidentally deleted some mail and i want it back. At the momment i can just copy those mails out of the snapshot and they are away within seconds.

Sorry about wording my post in a way that it seems to have offended but from an administration point of view our current setup seems to be a hell of a lot easier to maintain than zimbra and a hell of alot easier to scale which from my point of view is what you want when in a large environment.

So that is why i am trying to weigh up for myself will the migration be benifitial or not. And i thankyou all for your help because it seems i am finally starting to get some answers.

Woud it be possible for someone to allaborate on how you could restore mails to a dev server and then copy them over to the production? Maybe without even using snapshots. Cause i gather the mailbox would have to exist on the dev??

[Matt]

Your current setup is all accessing the same mailbox store which is equivalent to a zimbra mailbox server. You may have multiple frontend imap/pop3 servers but they're all reliant on the store. Perhaps you have drbd or a SAN solution for the store with a HA nfs server? I'm just trying to understand your current solution to compare with what's available with Zimbra.

To answer your last question; either by mounting the snapshot volume on a dev server, starting Zimbra and then using something like imapsync to copy the email. Of course it is also possible to restore a single mailbox on live (from daily backup) with a different name (so as not to overwrite the original) and then copy the mail between the two live mailboxes.