Mail stuck in deferred - 6.0.13 Logs indicate "Connection timed out"

[otgguys]

Our Zimbra install has been running great on Ubuntu 8.04 LTS & VMware server for over a year. We have upgraded a number of times and everything just works great.

Today I noticed a handful of emails in the "deferred" queue, some were emails I sent to my hotmail & yahoo accounts. The other emails were customers emailing their clients.

When I look in the zimbra.log I find the following..

Jul 12 08:46:49 zimbra postfix/smtp[19241]: E186F27E006: to=<user@verison.net>, relay=none, delay=300934, delays=300904/0.03/30/0, dsn=4.4.1, status=deferred (connect to verison.net[64.124.14.63]:25: Connection timed out)

Jul 12 10:17:59 zimbra postfix/smtp[21870]: 7FA1D27E004: to=<user@yahoo.com>, relay=e.mx.mail.yahoo.com[67.195.168.230]:25, delay=22, delays=0.04/0.01/11/10, dsn=4.4.2, status=deferred (lost connection with e.mx.mail.yahoo.com[67.195.168.230] while sending end of data -- message may be sent more than once)

Jul 12 10:27:40 zimbra postfix/smtp[10756]: 0322927E003: to=<user@hotmail.com>, relay=mx1.hotmail.com[65.55.92.168]:25, delay=1200, delays=0.02/0.01/600/600, dsn=4.4.2, status=deferred (conversation with mx1.hotmail.com[65.55.92.168] timed out while sending end of data -- message may be sent more than once)

Jul 12 14:45:51 zimbra postfix/smtp[19211]: 4B1B14797AB: to=<user@lawoffices.com>, relay=remote.lawoffices.com[67.79.127.54]:25, delay=14118, delays=14058/0.01/0.31/60, dsn=4.7.0, status=deferred (host remote.lawoffices.com[67.79.127.54] said: 451 4.7.0 Timeout waiting for client input (in reply to end of DATA command))

I have been all over the web looking for answers but everything I read would suggest I am fine. (Again everything worked until today and nothing changed not even a reboot)

-- Ok will I have rebooted after I descovered the problem but that was part of my troubleshooting :-)

NOTE: 1
I should note that emails sent to other domains hosted on this zimbra server are not deferred and I have no issues with incoming email.

NOTE: 2
I have been able to squeak out one email to my gmail account but one fell into this deferred bucket where it remains.

=============
My Config:
=============
HOST

/$ host `hostname`
zimbra.spamproofemail.com has address 192.168.1.50

DIG

zimbra@zimbra:/$ dig zimbra.spamproofemail.com mx

; <<>> DiG 9.4.2-P2.1 <<>> zimbra.spamproofemail.com mx
;; global options:  printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 761
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0

;; QUESTION SECTION:
;zimbra.spamproofemail.com.	IN	MX

;; AUTHORITY SECTION:
spamproofemail.com.	38400	IN	SOA	ns1.spamproofemail.com. admin.spamproofemail.com. 2006081401 28800 3600 604800 38400

;; Query time: 0 msec
;; SERVER: 192.168.1.50#53(192.168.1.50)
;; WHEN: Tue Jul 12 22:07:47 2011
;; MSG SIZE  rcvd: 89

==================================
==================================
zimbra@zimbra:/$ dig zimbra.spamproofemail.com all

; <<>> DiG 9.4.2-P2.1 <<>> zimbra.spamproofemail.com all
;; global options:  printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 36157
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 1, ADDITIONAL: 1

;; QUESTION SECTION:
;zimbra.spamproofemail.com.	IN	A

;; ANSWER SECTION:
zimbra.spamproofemail.com. 38400 IN	A	192.168.1.50

;; AUTHORITY SECTION:
spamproofemail.com.	38400	IN	NS	ns1.spamproofemail.com.

;; ADDITIONAL SECTION:
ns1.spamproofemail.com.	38400	IN	A	192.168.1.50

;; Query time: 0 msec
;; SERVER: 192.168.1.50#53(192.168.1.50)
;; WHEN: Tue Jul 12 22:08:59 2011
;; MSG SIZE  rcvd: 93

;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 17350
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;all.				IN	A

;; Query time: 4 msec
;; SERVER: 192.168.1.50#53(192.168.1.50)
;; WHEN: Tue Jul 12 22:08:59 2011
;; MSG SIZE  rcvd: 21

DIG of a domain where I am getting connection timeout.

zimbra@zimbra:/$ dig verison.net 

; <<>> DiG 9.4.2-P2.1 <<>> verison.net
;; global options:  printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 10817
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 13, ADDITIONAL: 0

;; QUESTION SECTION:
;verison.net.			IN	A

;; ANSWER SECTION:
verison.net.		67434	IN	A	64.124.14.63

;; AUTHORITY SECTION:
net.			150318	IN	NS	f.gtld-servers.net.
net.			150318	IN	NS	e.gtld-servers.net.
net.			150318	IN	NS	a.gtld-servers.net.
net.			150318	IN	NS	m.gtld-servers.net.
net.			150318	IN	NS	k.gtld-servers.net.
net.			150318	IN	NS	d.gtld-servers.net.
net.			150318	IN	NS	i.gtld-servers.net.
net.			150318	IN	NS	b.gtld-servers.net.
net.			150318	IN	NS	j.gtld-servers.net.
net.			150318	IN	NS	c.gtld-servers.net.
net.			150318	IN	NS	l.gtld-servers.net.
net.			150318	IN	NS	g.gtld-servers.net.
net.			150318	IN	NS	h.gtld-servers.net.

;; Query time: 0 msec
;; SERVER: 192.168.1.50#53(192.168.1.50)
;; WHEN: Tue Jul 12 22:10:38 2011
;; MSG SIZE  rcvd: 266

/etc/hosts output

127.0.0.1       localhost
127.0.1.1       zimbra.netgear.com      zimbra2
192.168.1.50    zimbra.spamproofemail.com       zimbra

Man if anyone has suggestions or ideas that can help me fix this I would buy you a Coke. :-)

[phoenix]

From the information you've posted it would appear you have no MX record for your Zimbra server for the domain you've checked. You are, however, checking the wrong domain information. This command is incorrect:

dig zimbra.spamproofemail.com mx

What you should be checking is this:

dig spamproofemail.com mx

You need to configure your DNS correctly. In addition to that your hosts file is incorrect, the entry for your loopback adapter is wrong. Go to the Split DNS article and read the details of what's needed for the DNS A & MX records and the correct format of the /etc/hosts file - you can check the records by running all the commands in the 'Verify...' section of that article.

If you connections are timing out then I'd guess you have a connection problem to the external mail servers, this can be caused by your firewall or a missing DNS server.

[otgguys]

Thanks Bill...

I looked at the Split DNS article and tweaked up my /hosts file name resolution looks good.

I started running network traces and found alot of retrans. I have a customer that is having issues connecting to the web interface as well as pop3.

I ran simultaneous network captures and not all packets are making it over to them :-(


We run Zimbra under VMWare Server so I swapped the NIC on the host machine, changed the cable and flipped over to another port on the server. None of this helped, more emails are being deferred. Most seem to be to hotmail, as I see in the logs emails to yahoo & gmail are making it.
The few hotmail emails that did ‘make it’ out of the queue never appeared in the hotmail accounts they were destined for. The emails that make it out to gmail\yahoo are visable in the mailboxs.

At this point I am moving the VM over to another server in the data center and I will retest to see if that makes any difference. This whole thing is so strange because one night it was working the next morning not-so-much.