650,000 Mailbox Migration Qmail-LDAP 2 Zimbra NE

[bonoboslr]

We have started our migration today successfully migrating 6000 accounts today from our qmail-ldap platform to Zimbra. We have not threaded our migration script yet, but unthreaded it migrates 1000 accounts / hour. We never used imapsync (too slow) but rather used external data source to pop the (non imap) mail to the Zimbra side.

Will keep updating here with details and look forward to any questions.

[mmorse]

Nice Scott!
9k down with 580,000 remaining? That makes 670,000 by my count

[Krishopper]

Just curious - how many servers does this environment have? MTA's? Mailbox? LDAP? Clustered?

The layouts of environments fascinate me.

[bonoboslr]

Originally Posted by Krishopper

Just curious - how many servers does this environment have? MTA's? Mailbox? LDAP? Clustered?

The layouts of environments fascinate me.

At this moment in time we have 4 clustered mailstores (16 cores each, 8Gbs of RAM). We have an 2 load balanced ldap slaves and at the moment 1 ldap master (soon to be clustered with another server) - masters have 12Gb RAM, slaves have 8Gb. We then have load balanced cluster of SMTPs and another load balanced cluster of Proxies (http,pop3,IMAP).

We will eventually have seperate logserver as well increasing the proxies and smtp servers as the need arises - currently 2 of each.

[bonoboslr]

Migration Status Bonobo’s Blog

update ^^

[bonoboslr]

Originally Posted by mmorse

Nice Scott!
9k down with 580,000 remaining? That makes 670,000 by my count

Mathematics not your strongest point?

[mmorse]

It was a joke to see if you'd catch it

Your blog said 9k:
580,000
009,000
=
589,000



So I assumed you meant 90k:
580,000
090,000
=
670,000

Which would still be 20k more than the thread title of 650,000

Either way congrats! (Just didn't want you to loose 20k or 61k users in the move count lol)

[bonoboslr]

The user list from ldap is ±580,000 - but the way we are growing we will be at the number below within 6 weeks.

$ zmlicense -p
AccountsLimit=650000

[marcg]

Thanks for sharing this info :-) I have a couple of additional questions if you are able/willing to share further:

Originally Posted by bonoboslr

At this moment in time we have 4 clustered mailstores (16 cores each, 8Gbs of RAM).

Do these use local disks or SAN? What RAID set-up? - any idea what your IOPS /user look like at this stage? What is your average user profile - I'm guessing, as you mention in your blog most are POP'ing, that it's "consumer, light use" profile?

Originally Posted by bonoboslr

We have an 2 load balanced ldap slaves and at the moment 1 ldap master (soon to be clustered with another server) - masters have 12Gb RAM, slaves have 8Gb. We then have load balanced cluster of SMTPs and another load balanced cluster of Proxies (http,pop3,IMAP).

It would be interesting to know exact numbers for the Proxies and SMTP boxes when you're running at full capacity and also if you're doing address verification, spam scanning etc. to get a real world idea of performance.

Good luck with the remaining users - however many there are left ;-)

[bonoboslr]

We are using SAN storage for index,redolog,logs,db/data,store,secondary_store. We have opted for maximum spindles as possible to increase performance. IOPs I will be able to give a better indication when we are in full swing. It will be negligible just now as we do not have that many users migrated across and they are all still using pop. Our intention is to change that though.

I cannot see us deploying more than 4 of each proxies or SMTP servers sun galaxy 4450s are phenominal beasts. Lots of disks, lots of cores, lots of RAM!!

zimbra_mail=# select count(*) from users;
count
-------
54423

Still quite a few!