Page 2 of 3 FirstFirst 123 LastLast
Results 11 to 20 of 28

Thread: Multi- server setup query

  1. #11
    Jesster's Avatar
    Jesster is offline Trained Alumni
    Join Date
    Nov 2007
    Location
    Santa Barbara, CA.
    Posts
    109
    Rep Power
    7

    Default

    We also started off running Zimbra on a single box (FOSS) then upgraded to NE, and decided we needed Multi Server. While we still have some work to do, here's our current setup:

    LDAP:
    We have 2 Linux servers handling LDAP. One is always master, while the other is replica. Should the master server die, we use Linux-HA heartbeat to move the virtual IP used for the master LDAP service over to the second server. We then use a modified ~zimbra/conf-master folder to start the master LDAP service against the same openldap-data replica data. We've only used this automatic system once or twice -- we try to avoid it if possible. The firewalls on these systems only allows LDAP port 389 and SSH 22 from other Zimbra servers in our Network.

    MTA:
    Our MTA's are split into two MTA roles: Relay and MX. Both systems use AntiVirus/AntiSpam from Zimbra. The MX servers handle inbound email only, while the Relays handle customer initiated outbound emails. The MTA MX's have SMTP Auth disabled, and the MTAMyNetworks is very minimal. The Firewall only allows port 25 connections in. We use twelve servers for MX purpose.

    For Relay, we use SMTP Auth as well as our IP networks listed in MTAMyNetworks. The firewall on the Relay servers allows traffic on ports 25 and 587. We have two Relay servers, each server's hostname matches the zimbra hostname, however, we also have Virtual IPs for each box that we point "smtp.example.org" to. This round-robin style DNS plus "Wackamole/Spread" open-source software allows us to become independent from any network load balancers.. (What if your Load Balancers fail?). Wackamole is a peer-based software that makes sure the VIPs are evenly distributed (as much as possible) and if any host drops, the VIPs will move automatically to other available servers. We opt'd not to do this for our MTA MX servers, because the DNS MX weights are resilient enough for a host failure automatically.


    PROXY:
    We use two Zimbra Proxy servers (no other Zimbra services installed) Similar to our MTA Relay, we use 2 extra VIP's for production hostnames such as webmail.example.org, pop.example.org, and imap.example.org. Remember these hostnames resolve to all VIPs that Wackamole/Spread maintain. The firewall here blocks everything except POP(s), IMAP(s), Webmail(443).

    Mailbox:
    We use two physical servers (no VM) with DAS external storage. We don't use the logger service or Anti Virus/Anti Spam (we relay all outbound mail to the MTA relays). For HA what we do use use Linux-HA heartbeat to manage a VIP that Zimbra binds to, as well as to manage the DAS /opt/zimbra mount points. We have an identical standby (a 3rd server) that has all the DAS's wired in. Should either of our two production Mailbox servers die, Heartbeat will bring up the VIPs and mountpoints. Our standby server can only be active for one mailbox server at a time. The firewall on the Mailbox servers only allows the other Zimbra servers to connect. No public access here. I should mention that there are domain attributes such as the ZimbraPublicServiceHostname that you should point to your proxy hostname (ie: webmail.example.org) so that REST urls do not point directly to your physical Mailbox Servers URL.


    Lately we've been switching off Xen and onto VMware ESXi for some of the non-Mailbox instances, but otherwise it's a very resilient system for us. **knocks on wood**

    Hope the info helps.

    --
    Jessie

  2. #12
    chandu is offline Elite Member
    Join Date
    Dec 2007
    Posts
    445
    Rep Power
    7

    Default

    Thank you so much J2b, L. Mark, Jessie for your inputs and time.

    Currently I have setup below environment :

    1. Ldap Matser server + slave server
    2. 1 MTA + proxy server
    3. Store 1 server with /opt/zimbra/db/data on ISCSI partition and
    /opt/zimbra/store on NFS partition

    This above setup is working fine without any issue.
    Now my next plan is as below :

    1. Prepared store 2 and make him the member of /opt/zimbra/store NFS partition of store1 for redundancy and prepared mysql slave on ISCI partition...i m just thinking ...need to test advanctage n disadvantage....even not sure reliability and performance of NFS parttion for store....this store 2 will keep same data as store1 ...

    2. I want to setup 2nd MTA server for MTA redudancy and scalability...I am not sure can we set outgoing traffic for 2 diffferent MTA...
    If its not possible then will allow incoming traffic on both MTA and set higher and lower priority and only outgoing traffic from MTA2 but here again the question of MTA2 redundancy...can any1 suggest on this...and can we enable 2nd proxy service here ?

    3. And yes i want to understand what is the best practice to setup DNS in multi-server environment..here on each VM, I have put the common entries of all domains and servers ...is it the correct method ? pls confirm.

    Thanks
    Last edited by chandu; 03-03-2010 at 02:14 AM.

  3. #13
    LMStone's Avatar
    LMStone is offline Moderator
    Join Date
    Sep 2006
    Location
    477 Congress Street | Portland, ME 04101
    Posts
    1,374
    Rep Power
    11

    Default

    I know some have had good success with NFS, but putting the store on an NFS share I confess gives me some pause.

    Any chance you can make that NFS share DASD or iSCSI?

    Also, Zimbra MySQL data is per-mailbox-server, so if you are going to set up a separate non-Zimbra MySQL replica, you'll need to run a separate MySQL instance on the MySQL replica server for each Zimbra mailbox server you have.

    Hope that helps,
    Mark

  4. #14
    j2b's Avatar
    j2b
    j2b is offline Special Member
    Join Date
    Sep 2008
    Location
    Latvia
    Posts
    165
    Rep Power
    6

    Default

    Congrats on your new setup!

    As to existing - I can not think of a reason why you would like to mount ./data and ./store on different mounts, but that could be ok, if you have other reasons to do so. As far as I understood Zimbra mailbox servers, to make them redundant for the same data, each server is wise to consider as full separate copy of another (even in OS layer), but you can not gain active/active redundancy on FOSS. It is achievable only on Network addition and with RedHat cluster file system support. Main reason below. Thus, to deal with two boxes, you have to get all data in sync and operate with VIP (virtual IP address) to connect to active master. The other server is down from zimbra perspective and just syncing data. If master gose down, zimbra service is started on slave server, taking over VIP. In this case, it really does not matter, whether you sync data between these servers via DRBDA or remout iSCSI shares, or use any other solution for that. It is just a question of time in data difference and the time zimbra services have to start on slave server.

    As to your plans:
    1. If you go out with mysql outside Zimbra, then you can have network performance issues. And there are still unsolved problems concerning MySQL write operation load balancing. In my previous posts I described why we decided to leave mysql on zimbra mailbox servers - due for performance. And, for example, if your number of accounts grow, you just put them on another zimbra mailbox server and do not overload mysql for that.

    And for shares, I would go with iSCSI solution anyway. As LMStone mentioned, NFS could probably work, but my concerns are in NFS as a protocol. It is protocol based share retaining the biggest problem for clustering - file locking and state control, not talking about protocol overhead in communications. If your zimbra box get quite bussy, there could be problems with performance due to protocol. And if in some faulty files remain locked (as used by master server), the second slave server will have only read access to them in the best situation. iSCSI is block based device over IP. For sure, there is some overhead due to network throughput (could be double links to increase it), and for IP protocol overhead. But at least it does not block files. You just unmount from one server and mount on another the same set of data, when slave mailbox comes in play. If your both mailbox server will be up and running, you have to deal with correct DNS things, as VIP will be on master server. If you do not deploy any clustering file system, I think, there could be some issues on running both mailbox servers at the same time. Our only problem is time, during which slave mailbox decides, that master is down (that is quite quick) and the biggest reason - starting zimbra services on mailbox takes quite a big time - up to 40seconds. May be other solutions could come in, but I do not have any.

    The other thing, is that NFS is stateless protocol, meaning in particular, that you can not control communication states - whether they are successful or not. And from it, data inconsistency questions may rise. But again, may be someone could comment on this issue more specifically and better describing failover operations and consequences.

    2. MTA redundancy. This I mentioned in my first posts. As far as I found out, on zimbra mailbox you can set only one outgoing MTA. Currently can not approve, whether it is only for WebMail client or for all communications together. And that was the case, when you put outgoing MTAs in Round Robin cluster and place load balancer in front of them. The incoming trafic balancing is done by DNS with priority or Round Robin with equal priority settings. Outgoing MTAs are with load balancer in front. Ex: smtp1.example.com, smtp2.example.com, ... smtpN.example.com and load balancer has DNS on one IP for just smtp.example.com. By this you define for zimbra that outgoing MTA is smtp.example.com and that does the thing. It load balances and can spread your loads as needed. That was offer, where I mentioned, that you can afterwards add outgoing SMTP servers as needed, without touching any zimbra configuration. Keep in mind, that DNS settings for smtpN servers should be correct, to deliver mail to gmail, yahoo, aol, etc. They have to have reverse zones, resovle to them and have TXT record for domains sending out mail, if different from server's hostname.

    Overall: if you have 4 MTA servers with one LB server, then configuration could be as follows:
    Incoming SMTP by DNS priority or (RR):
    mx1.example.com 10 (10)
    mx2.example.com 20 (10)
    ...
    Outgoing SMTP
    smtp.example.com - LB with vip configured in zimbra as outgoing MTA server providing load balance depending on your needs - Round robin, or other ways (depends on complexity of LB, but I think F5 could deal with it).
    smtp1.example.com - first SMTP
    smtp2.example.com - second SMTP
    ...

    Redundancy of LB servers could be achieved in several ways, depending on system you deploy. But there are solutions...

    3. DNS - to be honest, I forgot the main issues on that, but can look in former settings. But there were issues and they are very important. All zimbra servers are very DNS relyant and all settings should be correct. Otherway it will not start service or provide any communications. You can do it by defining hosts in local/external DNS servers, or, if you prefer, do it with /etc/hosts files, if your lookup is configuret default way, /etc/hosts file is checked first, and only then - DNS lookup is made.

  5. #15
    chandu is offline Elite Member
    Join Date
    Dec 2007
    Posts
    445
    Rep Power
    7

    Default

    Thanks J2b and Mark...That is really useful information...
    I got stuck on some other work so didnt get chance for further testing...

    Ok now i m trying to understand how proxy work in multi-store..as u know I am using open source edition and just checking out all possible ways for HA scalibility...so going step by step..

    I went through below link to set up zimbra proxy..I hope these r the only steps we need to perform regatrding the proxy...

    Enabling Zimbra Proxy - Zimbra :: Wiki

    But sorry to say i really couldnt understand how proxy will come to know on which server request should go as these commands only define communication ports on proxy server and store servers....

    There is no any document that how can we add new mailbox server in multi-server ...i googled some of the links and as per J2b's post we have to create COS per mailbox server and have to mentioned mailbox server in server pool and then while creating account have to mentioned respective COS....

    Is it right procedure ?? Mark and jessie, it will be great if u confirm this...i knw j2b, right now you have single store setup..

    Just had look on Mike's reply on below post :

    CoS, Domain, Server

    As he mentioned we can put multiple STORE server entires under server pool of DEFAULT COS. Does it means if I create new account by using default COS which contain all stores's entry in server pool,,then that particular account will be avalible and shared on ALL STORE SERVERS ??



    And ya i have another question also .....how can ldap will come to know about this additional store server ??? as all authentications are happeneing through ldap ...is there any setting need to do on ldap master once we add new store server ??


    Please help..

    Thanks
    Last edited by chandu; 03-07-2010 at 06:32 AM.

  6. #16
    j2b's Avatar
    j2b
    j2b is offline Special Member
    Join Date
    Sep 2008
    Location
    Latvia
    Posts
    165
    Rep Power
    6

    Default

    Hi, chandu...
    Shall come back with more info on your questions on proxy and ldap. But, just noticed in Zimbra Admin web console (probably that was implemented in 6.0.5, but didn't check that). There now is possibility to define several outgoing SMTP MTAs for web mail outgoing mail. Formerly there was only one field. Now, you can input several hostnames. Have not checked how does these work on failover or other MTAs problems, nor can confirm any load balancing stuff. But, tu simplify configuration, no LB for outgoing SMTP server stack is needed any more. At least, we have possibility to define several SMTP servers in Zimbra.

    Details later.

  7. #17
    chandu is offline Elite Member
    Join Date
    Dec 2007
    Posts
    445
    Rep Power
    7

    Default

    thanks j2b.thats is valuable information....i will definitly check out the Multiple MTA option in zcs 6.5..

    I have some more questions aboutr ldap

    1. is there any possibility to setup ldap master - master ??

    2. If not possible then in master - slave how can slave will become master if original master goes down ::
    Refered this link :
    http://www.zimbra.com/forums/install...tml#post115102
    Mark ...can you please tell me that command which u couldnt recall in this post

    Thanks

  8. #18
    chandu is offline Elite Member
    Join Date
    Dec 2007
    Posts
    445
    Rep Power
    7

    Default

    Hi Guys,

    Can anyone put some suggestions on post # 15 and 17 ??

    Thanks

  9. #19
    LMStone's Avatar
    LMStone is offline Moderator
    Join Date
    Sep 2006
    Location
    477 Congress Street | Portland, ME 04101
    Posts
    1,374
    Rep Power
    11

    Default

    Quote Originally Posted by chandu View Post
    thanks j2b.thats is valuable information....i will definitly check out the Multiple MTA option in zcs 6.5..

    I have some more questions aboutr ldap

    1. is there any possibility to setup ldap master - master ??

    2. If not possible then in master - slave how can slave will become master if original master goes down ::
    Refered this link :
    http://www.zimbra.com/forums/install...tml#post115102
    Mark ...can you please tell me that command which u couldnt recall in this post

    Thanks
    1. LDAP Master-Master is not (yet) supported in Zimbra.

    2. To confirm that Postfix can actually query multiple LDAP servers, look at the /opt/zimbra/conf/ldap-*.cf files and you should see the replica server(s) listed first, followed by the LDAP master last.

    If not, the Multi-Server Installation Guide has the proper procedure:

    Configuring Zimbra Servers to use LDAP Replica
    To use the replica LDAP server instead of the master LDAP server, you must update the ldap_url value on the Zimbra servers that will query the replica instead of the master. For each server that you want to change:
    1. Stop the Zimbra services on the server. Type zmcontrol stop.
    2. Update the ldap_url value. Enter the replica LDAP server URL
    zmlocalconfig -e ldap_url=”ldap://<replicahost> ldap://<masterhost>”
    Enter more than one replica hostnames in the list typed as ”ldap:// <replicahost1> ldap://<replicahost2> ldap://<masterhost>”. The hosts are tried in the order listed. The master URL must always be included and is listed last.

    Additional Steps for MTA hosts. After updating the ldap_url, rerun /opt/zimbra/libexec/zmmtainit.
    This rewrites the Postfix configuration with the updated ldap_url.


    Hope that helps,
    Mark

  10. #20
    chandu is offline Elite Member
    Join Date
    Dec 2007
    Posts
    445
    Rep Power
    7

    Default

    Hi Guys,

    Thank you all for putting your thoughts and suggetions on this thread.

    Now my current setup s as below :

    Configuration details :
    I have setup below servers :
    1.1.2.1 ---> ldap1.test.com --> LDAP Master
    1.1.2.2 ---> ldap2.test.com ---> LDAP slave

    1.1.2.3 ----> mail1.test.com ---> MTA + Store + mysql
    1.1.2.4 -----> mail1.test.com ----> MTA + Store + mysql

    Mysql master - master configuration has been set between 1.1.2.3 and 1.1.2.4
    /opt/zimbra/store ---> mail data location ---> shared on NFS partition between 1.1.2.3 and 1.1.2.4


    F5 ( network Load balancer) configuration is below :
    LDAP server pool ---> 1.1.2.227 ---> priority has been set for both LDAP servers
    MTA server pool -----> 1.1.2.226 ---> round robin has been set for load balancing between both MTA

    POC result :

    There was some issue with DNS setup which has been resolved and now load balancing and HA has been tested and its working as per logic.
    But I have found mail data inconsistency during testing.

    I found if any point of time there is delay in mysql replication then mysql setup get confused when they get request through MTA roudn robin setup for same mail ID. and replication get stopped.

    I am trysing to understand how zimbra mysql work and getting sync with LDAp.
    I was thinking to make common external mysql for both mail stores but after checking on net i found it wont be feasible.

    It will be great if anyone give me any trick on mysql replication... I m not at all good in mysql.

    Thanks

Page 2 of 3 FirstFirst 123 LastLast

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. dns in multi server
    By vikjava in forum Installation
    Replies: 17
    Last Post: 04-03-2009, 12:45 AM
  2. com_zimbra_asterisk on a multi server setup?
    By stephenwilley in forum Zimlets
    Replies: 0
    Last Post: 01-20-2009, 04:33 AM
  3. Error after installation
    By robsontuxlinux in forum Installation
    Replies: 13
    Last Post: 09-11-2008, 09:48 PM
  4. Moving from single to multi server
    By stephenwilley in forum Administrators
    Replies: 1
    Last Post: 06-11-2008, 09:52 AM
  5. Multi server install not working
    By Nassri in forum Installation
    Replies: 18
    Last Post: 08-17-2006, 09:23 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •