Zimbra on Netapp

[knabe]

We use Netapp for everything. The install docs say to not use RAID 5 (NetApp is RAID 4 DP), not to use SATA drives, and not to use NFS. Google shows a new NetApp/Zimbra partnership but I can't find any Best Practices for installing on NetApp. Do they require iSCSI (we don't have fiber)? Is NFS on NetApp ok for the binaries? For the mail store? Any doc links would be greatly appreciated.

[quietas]

Essentially the thought behnd not using RAID5 is that the access time of RAID1+0 is faster and still provides redundency. It's same with SATA vs SAS, SAS, iSCSI, Fiber, and such have a much lower latency for file access. This is due to the mail being stored as many millions of tiny little files.

That being said, I have 250 users running on a 4 drive SATAII 3.0gb/s system in RAID 1+0. It works quite adequately, faster with a pile of 6 SAS drives is planned next.

[knabe]

So our environment is RAID 4 Double Parity on SATA drives and I can chose between iSCSI or NFS over Gig copper. There are over 20 spindles in my raid group. I could have as many as 2500 accounts on the system.

[dNb]

Originally Posted by knabe

but I can't find any Best Practices for installing on NetApp. Do they require iSCSI (we don't have fiber)? Is NFS on NetApp ok for the binaries? For the mail store? Any doc links would be greatly appreciated.

So the short answer is for a long time Zimbra recommended against (to the point of saying they wouldn't support) an installation on NFS. We're in the same boat in that almost all of our data (certainly the important stuff) resides on Netapp disk. When I talked to them a while back I wasn't entirely clear on the source of this prohibition, but I got the feeling it came from being burned in the past from bad NFS client or server implementations. Search the forums for NFS and you should be able to find the thread where this was discussed.

iSCSI is supported (as I understand it), and we were going to start heading in that direction when I had people I trust warn me off of using it because the Linux drivers weren't always as solid as they should be. I'm also not enamored by having the data stored in a fashion I can't really look at from another machine as we could do if it just lived on NFS.

That's where things sat for quite some time until fairly recently when one of the Zimbra Professional Services engineers started to do some testing with using NFS. He wrote up a report on his findings (summary: works pretty well if you get all of your server and client parameters right). I haven't seen this report officially distributed any place, but if you check with your sales rep he or she might be able to dig it up for you.

We haven't yet tried to follow the recommendations in that report and do some testing, but that's something we do plan to do (or even engaging Zimbra PSA to help us to do it because this is something you don't want to get wrong).

Hope this helps.

-- dNb

[cre]

It all boils down to iops and Netapp Raid DP does a very good job at handling parity writes. Microsoft also recommends that you use raid 1+0 for Exchange environments but I have zero performance problems with 2000 + accounts on RAID DP. I currently run my zimbra environment 8000 + mailboxes on RAID 1+0 but i am looking at possibly moving to RAID DP storage on my Netapp.

Snipit from an Exchange storage setup requirement

RAID-DP
RAID-DP from NetApp is a proprietary implementation of RAID double parity for data protection. RAID-DP falls within the Storage Network Industry Association definition of RAID-6. RAID-DP is also a trademark of NetApp.

Unlike traditional RAID-6, RAID-DP utilizes diagonal parity using two dedicated parity disks in the RAID group. RAID-DP is also similar to other RAID-6 implementations in terms of the reliability metrics and its ability to survive the loss of any two disks; however, a third disk failure will result in data loss. Whereas current RAID-6 implementations incur an I/O performance penalty as a result of introducing an additional parity block, RAID-DP is optimized in terms of reducing read I/Os due to the way the NetApp controller handles parity write operations. Unlike other storage controllers that write changes to the original location, the NetApp controller always writes data to new blocks, thus making random writes appear to be written sequentially.

[knabe]

Are you using Fiber or SATA drives? Thanks to everyone for your input.

[cre]

I am using fiber drives on my Exchange environment but I am studying SATA for the Zimbra / Netapp side. If you have operations manager licensed for your Netapp that is a great place to look at how your storage is performing.

[Ian Forbes]

You can absolutely use iSCSI/NFS on a Netapp for Zimbra. I was involved with a Zimbra rollout for a University in Canada that went iSCSI on a FAS3040A. In that installation we had discussed also going NFS but Zimbra had told us that they didn't recommend it. I think that they were nervous about certain implementations of NFS, but Netapp NFS is solid. In any case we installed all binaries and stores over iSCSI.

Since then my company did our own testing with NFS. We decided to place /opt/zimbra (MySQL, OpenLDAP, Indexes, binaries, etc) on an iSCSI LUN and /opt/zimbra/store, /opt/zimbra/backup on NFS mounts. This gives us the flexibility of dealing with an NFS filesystem that we can grow on the fly with any disruption. Also gives us the ability to do file level backups to tape out of /opt/zimbra/backup.

The added benefit of mounting the message store and backup partitions on Netapp NFS volumes is that those volumes get snapshotted on a scheduled basis for added data protection. All in all there are just too many reasons not to take advantage of running Zimbra (store and backup partitions) over Netapp NFS.

Oh yah, we're running everything on VMware ESX vm's :-)

Cheers

[knabe]

I decided to install the binaries on a mirrored local drive (135gb avail) and put the store and the backup directories on NetApp via NFS. I have moved 20 users over to it and everything seems to be going very well so far.

[leog21]

Hi Knabe,

Can you update us on how things are going what your setup is now?

We are running zimbra on drbd right now but are considering putting it on a Netapp filter.

Thanks,
Leo