Server configuration--array distribution question

[ewilen]

(Assume RHEL.)

Reading the forum and other documentation, I'm seeing a bit of divergence of opinion on how to distribute Zimbra's data across multiple storage arrays.

It seems that that / and /opt need to be on a relatively fast storage (RAID 1 or 10 with fast drives), but I've read that splitting them up is advisable to avoid an I/O bottleneck. HSM and backup can go on relatively slow storage, but again there's an argument for putting them on separate arrays so that the backup will fail independently of the rest of the system.

However in practice, for a small-moderate installation (~100 users), having more arrays seems wasteful as you end up needing a lot of drives to achieve anything other than RAID 1. So I've basically seen two recommendations:

• just break out / and put everything else on a RAID 10 array--seems more expensive, all else being equal
• put / and /opt on a single RAID 1 array of fast drives, with everything else on slower/cheaper array(s)--possible performance hit (??) in exchange for economy

How much of a difference does it make, performance-wise, to put / and /opt on separate arrays?

[LMStone]

Hi Elliot,

For 100 users, a new server like an HP DL-360/380 will be more than adequate. The heavy disk I/O user is Amavis's temp directory (which we put on a RAM disk), especially if you are not doing any pre-filtering in front of the box via an edge MTA or a firewall doing things like A/V, A/S, RBL checks, etc.

There's a performance wiki article that lists the more disk-intensive directories and suggests these be on separate partitions. I don't think it's critical for an installation of this size on a single-server with DASD to have a separate array for / and for /opt.

I'd instead drop in 12GB of RAM and configure the box with several RAID1 arrays for:

• /
• /opt/zimbra/backup
• /opt/zimbra/store
• /opt/zimbra/index

If you are tight on budget, you can combine the first two arrays above.

Hope that helps,
Mark

[ewilen]

Thanks very much, Mark.

I can easily drop in an edge server to do RBL checks, if that will help. For now I prefer to use Zimbra's built-in amavis/clam/SA for content filtering, but I could also run ASSP on the edge server--it would just complicate things a bit.

That said, I gather the 12 GB RAM is for putting amavis temp on a RAM disk. Interesting!

BTW, I came up with usage data today. My users send about 460 emails/day, and they receive about 3200/day (combined internal/external). The latter does not include spam which is blocked by RBLs, but it does multiply-count emails which are sent to multiple users. Average mail store/user (online+offline) is around 1 GB--current setup is mostly POP, anticipated setup with Zimbra is IMAP for ~15 Mac users and ZCO for the rest. A good chunk of that mail store will consequently be pushed up to Zimbra to take advantage of the access-anywhere and sharing benefits of IMAP/ZCO/ZWC.

[ewilen]

Correction to previous data. The 460 sent mails only covers messages that leave our mail system--i.e. outbound to Internet.

Actual # of messages sent from my users is about 1600/day. Therefore about 1140/day are internal emails, some # of which are sent to multiple addresses.

[LMStone]

Correction to previous data. The 460 sent mails only covers messages that leave our mail system--i.e. outbound to Internet.

Actual # of messages sent from my users is about 1600/day. Therefore about 1140/day are internal emails, some # of which are sent to multiple addresses.

That's a pretty small inbound/outbound volume; you shouldn't need a separate edge MTA, but doing RBL checking with, say zen.spamhaus.org on the firewall in front of the server will definitely help keep the web interface snappy for end users. You could reduce the RAM to 10GB safely as well.

Also, you could get away with a single RAID1 array of big enough size with that low a mail volume and number of users if you are using a RAM disk. RAM disk sizing is a black art, but if you limited your message size to 25MB you should be fine with a 1GB RAM disk.

We have deployed previous-generation DL-360 G4 pizza boxes for clients on a budget serving 250 users with daily mail volumes of 20K messages and about 2GB-4GB per day of mail total in single-server mode, and performance was just fine. There was always a firewall in front however doing RBL checking.

Hope that helps,
Mark

[ewilen]

Thanks again. Our current server will do fine as an RBL-checker + inbound MTA relay. It checks against 20 RBLs without any trouble.

[LMStone]

Thanks again. Our current server will do fine as an RBL-checker + inbound MTA relay. It checks against 20 RBLs without any trouble.

20 RBLs is a lot of DNS activity. Most sites we have seen use 2-5 RBLs at most. After that, the remaining RBLs fail to catch anything new.

Glad I could help!

All the best,
Mark

[ewilen]

I know--but honestly, on the server I'm running now, it doesn't seem to affect performance to any detectable extent. Most users are running POP which may be a factor; if I use it as an edge MTA (or ASSP filter) there won't be any users at all. On any given day I do catch spam on pretty much all of those 20 RBLs, but once I have content-filtering in place (Zimbra with SA and/or ASSP), I'll trim the list.

Anyway you've given me some good food for thought, and I appreciate it.

[LMStone]

I know--but honestly, on the server I'm running now, it doesn't seem to affect performance to any detectable extent. Most users are running POP which may be a factor; if I use it as an edge MTA (or ASSP filter) there won't be any users at all. On any given day I do catch spam on pretty much all of those 20 RBLs, but once I have content-filtering in place (Zimbra with SA and/or ASSP), I'll trim the list.

Anyway you've given me some good food for thought, and I appreciate it.

One other tidbit FWIW, if you allow your users to do IMAP, that can be fairly resource-intensive on a Zimbra server. IMAP clients can open multiple, simultaneous and very chatty connections.

Do let us know how your deployment turns out by posting in the Zimbra Success Stories forum, OK?

All the best,
Mark

[ewilen]

Also, you could get away with a single RAID1 array of big enough size with that low a mail volume and number of users if you are using a RAM disk.

Mark (or anyone else listening in),

How about a single RAID 10+hot spare array composed of 5 250 GB SATA drives @ 7200 RPM?

(In answer to your comment above, I anticipate about 75 ZCO users and about 15 IMAP users, though nothing will prevent people from using ZWC. Also I would guess around a couple dozen ActiveSync users on a combination of Fetch and Push.)

Do let us know how your deployment turns out by posting in the Zimbra Success Stories forum, OK?

I'll definitely keep you updated!