IMAPStore Replication/Failover IDEA - Multiple Virtual message stores

[kbaker]

R A I S - F S

IMAPStore - Redundant Array of Inexepensive Servers File System


So I've been in search of a good failover solution for an IMAP server for some time now. Basically to have a somewhat realtime hot backup of my server that I can failover to.

The problem is alway replicating the file system in some way while keeping the header and envolope information replicated to a remote server.

So the idea is to create a sort of RAID-1 array across multiple servers for a message store. Sort of a simple GFS...

I was thinking that we could pretty easily build a system where all the message "writes" to filesystem could actually be written to multiple machines. Their locations would be stored in a simple association table to the message-id. Then any request for a message would just do a quick lookup to find the message store. Obviosly this is a bit simplified.

- store a message

• store message file on multiple backend message stores 2 to n
• store message physical location (messageid, server, path) in meta db
• add message stores as storage is running out

- get message

• request message via messageid
• lookup messageid in meta db to get physical location
• grab message and deliver back into imap server exactly as it is now

The idea come from mass image storage systems and Google File System. These work in a similiar way. Essentially any file is stored in multiple locations... its location information is stored in a database. When the file is requested it finds the file in storage and returns it. It could be on one machine or hundreds.

If a system goes down the database with meta location information has already been replicated via db replication. When the file is requested from a server that is down it just grabs the next record for the "other" location on another server.

Why?

• Can add infinite number of servers to expand storage
• Can replicate to multiple data centers pretty easily
• Very cheap implementation, all software no SAN or expensive hardware
• Total Failover.... can setup monitor that assures that messages are
  replicated to as many machines as you'd like
• Tested, this is how we developed the mass-image storage at our company. This is essentially how Google File System works minus splitting files into blocks.
• Should be very pluggable into Zimbra... meaning make a chance to the MessageStore object and be done with it... the rest of the system shouldn't have a problem.

Anyone interested...?

[KevinH]

Interesting... Slightly different approach than our planned replication which will involve journal shipping. The advantage of our approach is you get all the meta changes and all the rest of the collaboration data updates/writes. So it's an entire solution and not just for email.

[Dirk]

I've looked into this sort of thing too, I considered using Network Block Device ( http://nbd.sourceforge.net/ ) which is essentially having a raid array split between disparate computers.

So you could have two servers, each with a set of drives set up in raid 5 via hardware, then you could use the NBD driver to mirror those so they were always in sync, all writes would end up on both sets of drives.

Anyway, I tried it, and couldnt get it working, your mileage may vary. I think I'd prefer to wait and see what the Zimbra crew come up with. Hacking your way under zimbra to make it fault tolerant is one thing, but having zimbra do it itself has to be a better way.

[goetzi]

Originally Posted by KevinH

Interesting... Slightly different approach than our planned replication which will involve journal shipping. The advantage of our approach is you get all the meta changes and all the rest of the collaboration data updates/writes. So it's an entire solution and not just for email.

That replication feature sounds very interesting. Will there be the possibility to replicate just a part of the mailboxes/users? Can you say that user group A will be synced to server X and group B to server Y?

[KevinH]

Not sure on that.. We've not completed the design.