Page 1 of 3 123 LastLast
Results 1 to 10 of 28

Thread: Multi- server setup query

  1. #1
    chandu is offline Elite Member
    Join Date
    Dec 2007
    Posts
    445
    Rep Power
    7

    Arrow Multi- server setup query

    Hi Guys,

    I am looking for one complete solution from where I can achieve zimbra scalability, performance and high availability.

    I have referred multi-server concept note and found it very interesting. Right now we have single zimbra mail server with 3 customer’s domains. So thinking to migrate my single setup to multi-server setup. But before that want to understand this setup completely.

    First I would like to concentrate on scalability and then HA. I am thinking about setup like below:

    1. LDAP master and replica server + zimbra mysql (2 VMs)
    2. Two MTA servers + antivirus + spamassessin (2 VMs)
    3. Two mailbox store servers (2 physical servers)

    I had rough Idea how can do basic installation for each setup. But Now I have below queries:

    1. Zimbra proxy: Where we need to enable zimbra proxy on above setup? I think it should be integrated with LDAP so it will come to know where to redirect the request through MTA ..Pls confirm.

    2. Zimbra MTA: I want such a solution which can handle email traffic smoothly during peak hours also. So how to setup multiple MTA for it? Can I define one MTA for incoming and another for outgoing traffic? or both MTA can handle both type traffic ? What will be the role of zimbraMailTransport parameter in such setup? or can I use DNS load balancing feature here ?


    3. As I mentioned proxy will decide on which mailstore the data should go but as both mailstore servers are individual so off course mail data and mailbox will be different on both servers....So how can we manage that? Do we need to segregate this mail data on domain base? What is the best solution you suggest for multi-store server setup?

    Please help and suggest.


    Thanks
    Last edited by chandu; 02-12-2010 at 02:26 AM.

  2. #2
    j2b's Avatar
    j2b
    j2b is offline Special Member
    Join Date
    Sep 2008
    Location
    Latvia
    Posts
    164
    Rep Power
    6

    Default

    Here are my considerations, based on 2 years work with OS edition and dealing with multiservers and DRBD block device mirroring.

    My current setup:
    1 LDAP server master (only) on 2 physical servers (HA, DRBD - active/passive)
    1 Mailbox server on 2 physical servers (HA, DRBD - active/passive) - Logger was here.
    2 MTA servers with Antispam/Antivirus (only one of them specific outgoing SMTP).
    1 Zimbra Proxy on 2 physical servers (like above).

    As to yours intended configuration...
    1. I do not know, whether it is possible to make separate (specific) location for Zimbra MySQL server, at least Zimbra recommended way. If you take MySQL out of MailBox server, you can get a bottleneck and network connection problems connectiong to it. So, I would leave MySQL with Mailbox to make fast queries locally. And it will be wasier deal with backups and recovery - one place.

    2. Currently I do not have this, but considering changes. I would strictly separate incoming SMTP and outgoing SMTP servers. At least 2 for any direction. Considering switching off AntiSpam and AntiVirus on outgoing servers, as they are servicing only Zimbra accounts so increasing sending procedures run by web client. An the other - I am considering on moving Zimbra SMTP servers deeper inside and configuring a cluster of SMTP Postfix servers in front row as for incoming, and outgoing traffic. By default, you can set only one outgoing smtp server for webmail client servicing, so if this server is down for any reason - nothing happens and users can not send out mails. It it is trusted environment, I do not think, that you have to make checks on your users' outgoing trafic and it could be done with pure postfix.

    3. Mailstore - my biggest problem is to achieve two active mailbox servers for a single mailstore. Currently we are working with drbd setup and it is going smoothly and stable. But switchover to another server takes about 3-5 minute to full zimbra start. And if HA decides to switch back to server 1 it starts to play backwards - thus increasing this downtime. By that time several things could happen with your mail, and much safer feelings would be, if you have really separate server, to wich route the users and leave them access to latest current mailbox data. As well such solution could help to make upgrades more transparent and users would not have to wait with downtime.

    As to your questions:

    1. Yes, Proxy has to get access to LDAP. For webmail proxying we use Apache Revers proxies, but considering to switch over for more specific nginx configuration and implementin caching to boost webmail performance.

    2. MTA - I would say, that this could be wise to separate MTAs by traffic direction. Zimbra MTA (if configured so) does not only pass SMTP traffic, but makes SpamAssassin/DSpam and ClamAV scanning too. And if attached files are quite large, it takes a time and resource for them to process. By this issue, we consider moving out AV/AS scanning to separate servers and do SMTP traffic by pure Postfix in cluster setup with DNS Round Robin for incoming mail and VIP (Virtual IP) for a set of outgoing servers by load balancer (actually the same round robin, only as you can configure only one outgoing address, it solves this and gives access to as many SMTP servers as you need, according to your load). And MTAs from Zimbra have to connect to LDAP too, to get info on recepient address. Thiss we are planning to address with a line of incoming SMTP servers registered only for specific domains and not checking exact e-mail address of the user, thus sacrificing their resources, but such servers will be purely simple, and if one goes down, you can boot another one in a minutes. And only after them pass mail to Zimbra SMTP incoming servers.

    3. Currently we do deploy only one mailbox server, so could not comment on this much. But Server pool is defined by COS settings and I think, that you can manage one domain on both mailstore servers, as COS is set for the user account. May be some one else could comment on this more deeply. But it could be correct, because, if, for example, as a ISP you hoste several thousands of emails in one domain, then it is hardly to believe, that it is done on one server.

  3. #3
    chandu is offline Elite Member
    Join Date
    Dec 2007
    Posts
    445
    Rep Power
    7

    Arrow

    Hi J2b,

    Thank you so much for your time and suggestions. That was really helpful.

    I am thinking about architecture as below :

    ################################################## #

    Incoming traffic flow :

    Internet --> Firewall --> Master LDAP + proxy server --> F5 netwpork Load balancer --> Incoming MTA Server + AV + SpamAssasin --> One of the mail store


    Outgoing traffic flow:

    One of the mailstore --> Outgoing MTA server WITHOUT AV and spamassasin --> F5 network load balancer ---> Firewall --> Interent

    ################################################## #

    Here I am confuse about Zimbra proxy..do we need to install zimbra proxy on LDAP server or there should be seperate servers we need to assign ?

    And how to achive multiple stores for one domain ?

    Is above mention traffic sequence is correct ? pls have a look on F5 place ...and ya our customers are using MS outlook and webmail for mail communication so have to consider https, imap/pop with/iwithout ssl traffic.

    Please help me to understand regarding proxy + multi-store setup.

    Thanks
    Last edited by chandu; 02-14-2010 at 10:35 AM.

  4. #4
    chandu is offline Elite Member
    Join Date
    Dec 2007
    Posts
    445
    Rep Power
    7

    Default

    Can anyone suggest me regarding above setup ?

    Thanks

  5. #5
    j2b's Avatar
    j2b
    j2b is offline Special Member
    Join Date
    Sep 2008
    Location
    Latvia
    Posts
    164
    Rep Power
    6

    Default

    Hi, to my concern, there are several ways to setup systems as far as you get control on them and feel comfortable in management area and issue solving tasks (and/or corporate policy). In all of the setups I follow my personal guidelines: with computer systems there is no question IF something will break or go down, but WHEN?! In such situation you have to deal with exact tasks to bring this system up as quick as possible, and if it involves any kind of testing or solution finding during recovery, then usually it takes more time and your users will be waiting for that!!! It all depends on downtime you can admit.

    For that said and looking on your plan for Incoming traffic, I would take OUT LDAP server (especially Master LDAP) from first layer and public addresses. This is where all your accounts are stored and this server should be guarded as much as possible. In our configuration, LDAP and Maistore servers are hidden even in third layer behind internal firewalls. Although, it matters on exact FW settings and policies. The only servers we allow on front row are Zimbra Proxy (for IMAPs/POPs access), Other reverse proxies for web access to mailbox server and a couple of SMTP (Zimbra MTA) servers.

    Concerning load balancer - if you have one, then it is OK, but still it does not give you pure high availability, if you would like to spread MTAs arround different subnets or different locations (data centers), which could be wise anyway. I would do incoming SMTP based on DNS Round Robin settings for LB or just pointing MX records with different priorities. But thus anyway, if you would like to stop all your wrong e-mail address posting delivery, you have to give access to LDAP for all your MTAs. My former suggestion was play with overhed in system inefficient usage by servicing mails for non-existing e-mails and configure layer 1 SMTP servers manually for domains only and not connect them to Zimbra LDAP. But it all depends on your project spread - how many user boxes you will be servicing and how much the domains will differ.

    Why bother with LDAP as such? We had a problem while using only one LDAP server. Yes, it was placed on cluster with drbd, but if server does not considers, that master went down, it didn't switch over to slave. And problems with corrupted LDAP database was such. Our main LDAP stayed online, but no services were provided by it. So in this case you have to switch quickly for another LDAP server, otherwise no Zimbra services will be offered at all, even if you have several HA provisions for other Zimbra servers. By this - I would expose to services LDAP slave servers, but Master LDAP shoud be used by Mailbox servers on write operations and accounts' registration. Please explore Zimbra architecture.

    Designing paths and information flow, you have do look through all provided services separately, to not mix or complicate firewall settings. And it does not matter technically on which servers you install which Zimbra sub servers together, meaning, you could install all Zimbra stack on one server, or every Zimbra sub server (LDAP, MTA, Proxy, etc.) on separate on. The difference is in logic and security assumptions.

    Taking in mind above, please revize atleast another architecture:
    Internet -> Firewall 1 (First layer - public addresses)
    Here proxy and MTAs should be placed
    Proxy/MTA -> Firewall 2 (Second layer - Internal servers)
    Here LDAP and Maibox could go in.

    Again - LDAP - the best it could be as it is one of the slaves. Although (concerning LDAP Master/Slave) haven't look through this configuration yet but plan to reorganize our stack this way. In this situation, if on of your LDAP (Slave) servers will go down for any reason, you could quickly switch over to another, but your main data and directory on LDAP Master remains untouched. From there you provision your accounts.

    The outgoing traffic schema could be OK, except that Load balancer should be used before outgoing MTAs stack, because up till now I could not find a solution to allow zimbra mailbox server to send outgoing mails through several (primary, secondary, etc.) SMTP servers. The configuration implies to set up only one SMTP server for outgoing traffic. May be it is possible to input csv value with comma? Didn't have a time to experiment. In such case LB will provide address like smtp.company.com for zimbra mailboxes, but behind it, it will spread the load between smtp1., smtp2., etc. servers. And you can add these servers as you need, depending on the overall load. I do not see any use of placing load balancer behing the outgoing SMTP servers.

    And for overall load:
    1. Again, I do not know the planned amount of traffic and mailboxes and possible attachment settings. But one for sure - as bigger attachments on mails you allow, the more resources of AV server will be used. As we allow quite large attachments (due to our customer requests), we intend to rework the setup and take out AV/AS servers out from first layer incoming SMTP servers, to decrease load and process more quickly mails without attachments. Such system gives you another flexibility - you can plug in other AV/AS solutions as well, even if you intend to go for commercial ones.

    2. Logger - Logger takes resources, and as to Zimbra Multisever recommendations, it has to be installed on Mailbox servers. To process web access requests more quickly, I would install separate mailbox server ONLY for administrative and logger statistics purposes.

    Your questions:
    LDAP + Proxy - No, you do not need to install them together, but still can do it. From security point of view, I would leave Proxy alone and put LDAP near mailbox on the same, or better on different subnet/vlan. The other is firewall configuration and routing.

    Multiple stores for one domain: As I've told before, didn't managed to test or deploy it, but as such settings are in COS setup - Server pool tab. Each COS can be applied to different users, which defines, which user is using what. And by this I assume, that this is the way to look for info. But as COSes are not domain specific, may be the registration process on one or the other mailbox server has to take place manually. Although may be some one else can comment on this. Sorry.

    Traffic sequence: As mentioned in this reply, let's separate specific services and discuss on them separately - meaning: Incoming SMTP, Outgoing SMTP (as well if you provide your customers authenticated SMTP server usage for outgoing emails from their mobile phones or laptops), LMTP (internal mail SMTP transfer from MTA to mailbox server), WEB (http/https). IMAP/POP, etc.

    As for F5 - for incoming SMTP - do not see any necessity, as it could be done more wise with DNS. Outgoing - no - set it before SMTP servers, not behind.

    Proxy & multistore: For this Zimbra Proxy should be used or you can play with nginx proxy by your self. But main tasks Zimbra Proxy is doing, is proxying Web, IMAP, POP traffic. Web traffic could be proxied with any other load balancers or Reverse proxies, but for IMAP and POP Zimbra proxy find exact mailbox location via LDAP - e.g. on which mailstore server exact mailbox is located, and redirect user to specific mailstore server. Probably failover and load balancing on such proxy could be an issue too, depending on the overall load and configuration you intend.

    May be it could be easier for you to look on all systems as separate servers, deal with traffic, and afterwards decide which of them could be combined in one machine.

  6. #6
    LMStone's Avatar
    LMStone is offline Moderator
    Join Date
    Sep 2006
    Location
    477 Congress Street | Portland, ME 04101
    Posts
    1,373
    Rep Power
    10

    Default

    The Zimbra docs recommend installing proxy on an MTA.

    Our view is that doing so is a good idea; it enables you to put a firewall between your MTAs/Proxy and your mailbox servers.

    Sorry if I missed it, but running an LDAP replica on your MTAs (or a separate LDAP replica) will help keep your mailbox servers and the Admin Consoles running snappily, even when your system is getting hammered with spam.

    Hope that helps,
    Mark

  7. #7
    j2b's Avatar
    j2b
    j2b is offline Special Member
    Join Date
    Sep 2008
    Location
    Latvia
    Posts
    164
    Rep Power
    6

    Default

    Proxy on MTA - probably could go, but if MTAs are very bussy or you would like to spread the load, than I think it is easier to deal with replacing only one SMTP system, than deal with additional configuration. But it could depend on power of the servers, I guess.

    In our case one MTA which is for incoming/outgoing SMTP server gets pretty busy, and we have to allow quite large attachments. While monitoring this server - it seems that overall load is getting for such attachments' antivirus scanning process, during wich sending mails suffer decrease on Send operations, while using WebAccess client. And may be porxy as separate server could be too lightweight to run on separate server.

    LDAP replica with MTA on same box - could be reasonable for performance, but how about security of LDAP as requests could be done anonymously? If it is behind the firewall, you atleast can restrict access from specific MTAs, or if MTA server is compromized?

  8. #8
    LMStone's Avatar
    LMStone is offline Moderator
    Join Date
    Sep 2006
    Location
    477 Congress Street | Portland, ME 04101
    Posts
    1,373
    Rep Power
    10

    Default

    If you put Amavis's temp directory on a RAM disk you will find your MTA servers will be very fast.

    If your MTA server is firewalled to allow only port 25 traffic inbound then LDAP should be safe.

    If you perform RBL checking in front of your MTA then LDAP on the MTA server will be even safer, since 85%-95% of the inbound connections will be dropped before they reach the MTA.

    Hope that helps,
    Mark

  9. #9
    j2b's Avatar
    j2b
    j2b is offline Special Member
    Join Date
    Sep 2008
    Location
    Latvia
    Posts
    164
    Rep Power
    6

    Default

    Your tips sound reasonable. Just read your article from year 2007. Amavis RAM Disk Setup - OK Now To Reimplement?

    I am just suspicios concerning controlling connections inside one server. And try to build systems, where I can control interconnections between them by firewall means.

    And thank you for the tip.

  10. #10
    LMStone's Avatar
    LMStone is offline Moderator
    Join Date
    Sep 2006
    Location
    477 Congress Street | Portland, ME 04101
    Posts
    1,373
    Rep Power
    10

    Default

    Quote Originally Posted by j2b View Post
    Your tips sound reasonable. Just read your article from year 2007. Amavis RAM Disk Setup - OK Now To Reimplement?
    And it still works; we have been using it continuously since that post with no problems whatsoever -- other than Zimbra once in a while moving directories around...

    Quote Originally Posted by j2b View Post
    I am just suspicios concerning controlling connections inside one server. And try to build systems, where I can control interconnections between them by firewall means.

    And thank you for the tip.
    I understand your suspicions. We think of Zimbra as an appliance where you get to choose the underlying operating system, and having read/filed a number of bug reports, it's clear Zimbra does a lot of inter-component QA work.

    If you look again at the Zimbra architecture documentation, the performance tuning for large systems wikis and the multi-server guides, you'll see that Zimbra's various components can be distributed across a number of servers, allowing you to place hardware firewalls between the servers if you like.

    You can put a Zimbra proxy server for example in one DMZ, your MTAs in another DMZ and leave the mailbox servers on the LAN zone. That starts to get a bit complicated, but if your security requirements dictate something that hardened, Zimbra can do it.

    IMHO, managing security is all about managing risk. And reducing risk costs money and time. At some point, the costs to reduce risk just a little bit more become prohibitive. But that point is very different for different companies. And minimum standards for risk reduction (and therefore costs) are for many companies set by regulatory requirements and/or industry-specific accepted best practices.

    That's a long way of saying that how much security you'll need to include in your Zimbra installation will likely be very different from many other Zimbra installations, and that's OK.

    Hope that helps!

    All the best,
    Mark

Page 1 of 3 123 LastLast

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. dns in multi server
    By vikjava in forum Installation
    Replies: 17
    Last Post: 04-03-2009, 12:45 AM
  2. com_zimbra_asterisk on a multi server setup?
    By stephenwilley in forum Zimlets
    Replies: 0
    Last Post: 01-20-2009, 04:33 AM
  3. Error after installation
    By robsontuxlinux in forum Installation
    Replies: 13
    Last Post: 09-11-2008, 09:48 PM
  4. Moving from single to multi server
    By stephenwilley in forum Administrators
    Replies: 1
    Last Post: 06-11-2008, 09:52 AM
  5. Multi server install not working
    By Nassri in forum Installation
    Replies: 18
    Last Post: 08-17-2006, 09:23 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •