Sudden Outgoing DNS (MX) Lookup failure

[wdimmit]

After working normally for a week or so, my Zimbra install has decided to stop doing proper MX lookups for outgoing mail, leaving me unable to send mail to any domain who's main IP (domain.com) is different then it's mailserver's IP (whatever host listed in MX record). My nameservers are working - 'dig MX domain.com' returns the correct information. An example is gmail. Postfix reports timeouts trying to connect because it is trying to connect to the IP referenced by gmail.com. This is M2 running on CentOS 4.

--Will

[marcmac]

Did you do any OS updates?

[wdimmit]

Nope. As far as I can tell, I didn't do anything of any signifigance between when it worked and when it stopped working. Any ideas where I might look for a problem?

[marcmac]

It's a postfix problem, so check the postfix logs.

No packages on the server were updated in the intervening time? Was the server rebooted?

[wdimmit]

No packages were updated, although in an attempt to fix this I did uninstall my separate (not running) copy of Postfix, restarted Zimbra a few times, restarted the network interfaces, insured that nscd was NOT running, switched to new DNS servers and rebooted the server. The postfix log entries seem clean, its just descided to do incorrect lookups. Is there anything Zimbra-specific in the Postfix setup that could affect this?

Thanks,
Will

[marcmac]

This could be a library issue. The postfix that you uninstalled - was it a package, or something you compiled yourself? (Which OS are you on?)

Do this:
ps auxww | grep postfix
Find the "master" process
lsof -p <pid of master process>

Which libraries is it using?

Also, ldd /opt/zimbra/postfix/libexec/master as root may help.

[wdimmit]

The postfix I uninstalled was a RPM package (postfix-2.1.5-4.2.RHEL4) and was uninstalled AFTER the problem started.

'ldd /opt/zimbra/postfix/libexec/master' returned:

        libpcre.so.0 => /lib/libpcre.so.0 (0x00964000)
        libldap-2.2.so.7 => /opt/zimbra/lib/libldap-2.2.so.7 (0x00202000)
        liblber-2.2.so.7 => /opt/zimbra/lib/liblber-2.2.so.7 (0x00527000)
        libmysqlclient.so.14 => /usr/lib/mysql/libmysqlclient.so.14 (0x00c64000)
        libz.so.1 => /usr/lib/libz.so.1 (0x00952000)
        libm.so.6 => /lib/tls/libm.so.6 (0x00917000)
        libsasl2.so.2 => /opt/zimbra/cyrus-sasl/lib/libsasl2.so.2 (0x0086a000)
        libpthread.so.0 => /lib/tls/libpthread.so.0 (0x009a3000)
        libssl.so.4 => /lib/libssl.so.4 (0x00c2e000)
        libcrypto.so.4 => /lib/libcrypto.so.4 (0x00b43000)
        libnsl.so.1 => /lib/libnsl.so.1 (0x00a53000)
        libresolv.so.2 => /lib/libresolv.so.2 (0x009c1000)
        libc.so.6 => /lib/tls/libc.so.6 (0x0022e000)
        libcrypt.so.1 => /lib/libcrypt.so.1 (0x00111000)
        /lib/ld-linux.so.2 (0x007d2000)
        libdl.so.2 => /lib/libdl.so.2 (0x0094c000)
        libgssapi_krb5.so.2 => /usr/lib/libgssapi_krb5.so.2 (0x00b2d000)
        libkrb5.so.3 => /usr/lib/libkrb5.so.3 (0x00aa3000)
        libcom_err.so.2 => /lib/libcom_err.so.2 (0x00946000)
        libk5crypto.so.3 => /usr/lib/libk5crypto.so.3 (0x00b0a000)

And did not seem to change anything.

The relevant info from 'lsof -p' is:

master  1711 root  mem    REG        3,3   27191 4310089 /lib/libcrypt-2.3.4.so
master  1711 root  mem    REG        3,3   45800 4310658 /lib/libnss_files-2.3.4.so
master  1711 root  mem    REG        3,3  619147 6425359 /opt/zimbra/lib/libldap-2.2.so.7.0.21
master  1711 root  mem    REG        3,3  106397 4309984 /lib/ld-2.3.4.so
master  1711 root  mem    REG        3,3 1454462 4325450 /lib/tls/libc-2.3.4.so
master  1711 root  mem    REG        3,3  178019 4325459 /lib/tls/libm-2.3.4.so
master  1711 root  mem    REG        3,3    7004 4310112 /lib/libcom_err.so.2.1
master  1711 root  mem    REG        3,3   15324 4310065 /lib/libdl-2.3.4.so
master  1711 root  mem    REG        3,3   63624 5046338 /usr/lib/libz.so.1.2.1.2
master  1711 root  mem    REG        3,3   63640 4309991 /lib/libpcre.so.0.0.1
master  1711 root  mem    REG        3,3   93865 4325460 /lib/tls/libpthread-2.3.4.so
master  1711 root  mem    REG        3,3   79488 4310100 /lib/libresolv-2.3.4.so
master  1711 root  mem    REG        3,3   95148 4310128 /lib/libnsl-2.3.4.so
master  1711 root  mem    REG        3,3  253975 6472803 /opt/zimbra/cyrus-sasl-2.1.21.ZIMBRA/lib/libsasl2.so.2.0.21
master  1711 root  mem    REG        3,3  415188 5047461 /usr/lib/libkrb5.so.3.2
master  1711 root  mem    REG        3,3  136016 5047460 /usr/lib/libk5crypto.so.3.0
master  1711 root  mem    REG        3,3   82944 5047583 /usr/lib/libgssapi_krb5.so.2.2
master  1711 root  mem    REG        3,3  936960 4310082 /lib/libcrypto.so.0.9.7a
master  1711 root  mem    REG        3,3  213600 4310636 /lib/libssl.so.0.9.7a
master  1711 root  mem    REG        3,3 1139172 5181551 /usr/lib/mysql/libmysqlclient.so.14.0.0
master  1711 root  mem    REG        3,3  125060 6425354 /opt/zimbra/lib/liblber-2.2.so.7.0.21

Thanks,
Will

[marcmac]

Do you have dns lookups enabled in your MTA?

[wdimmit]

*sheepish* That'd be the problem...I misinterpreted the meaning of that checkbox.

Thanks,
Will