internet outbound mail stuck in the queue

[zacheus]

I have a new installation of zcs 7.1.3 with split dns service on the same host(ubuntu 10.04). I have counter-checked my configurations with those on split dns wiki.

dig & nslookup commands show my internal dns is ok. I can as well dig & nslookup internet domains.

Problem
1. when I ping an internet domain, it resolves the correct public ip but does not reply the icmp messages. it hangs after resolving the public ip.

2. Local mail delivery within the domain accounts is working ok but when I try to send internet mail eg to gmail, yahoo, hotmail etc. the emails get stuck in the queue as deffered.

I tail-ed the /var/log/zimbra.log while sending the mails and it says connections timed out. Any help accorded will be highly appreciated.

additional information is as below:

.................................................. .................................................. ....

root@mail:/home/optiplex# dig -x mosby.com

; <<>> DiG 9.7.0-P1 <<>> -x mosby.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 7044
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0

;; QUESTION SECTION:
;com.mosby.in-addr.arpa. IN PTR

;; AUTHORITY SECTION:
in-addr.arpa. 3256 IN SOA b.in-addr-servers.arpa. nstld.iana.org. 2011024469 1800 900 604800 3600

;; Query time: 957 msec
;; SERVER: 192.168.1.3#53(192.168.1.3)
;; WHEN: Tue Jan 3 14:35:59 2012
;; MSG SIZE rcvd: 108

.................................................. .................................................. ....

root@mail:/home/optiplex# dig mosby.com any

; <<>> DiG 9.7.0-P1 <<>> mosby.com any
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 778
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 1

;; QUESTION SECTION:
;mosby.com. IN ANY

;; ANSWER SECTION:
mosby.com. 604800 IN SOA mail.mosby.com. admin.mosby.com. 70729 604800 86400 2419200 604800
mosby.com. 604800 IN NS mail.mosby.com.
mosby.com. 604800 IN MX 10 mail.mosby.com.
mosby.com. 604800 IN A 192.168.1.3

;; ADDITIONAL SECTION:
mail.mosby.com. 604800 IN A 192.168.1.3
.................................................. .................................................. ....

root@mail:/home/optiplex# nslookup mosby.com
Server: 192.168.1.3
Address: 192.168.1.3#53

Name: mosby.com
Address: 192.168.1.3
.................................................. .................................................. ....

root@mail:/home/optiplex# dig gmail.com any

; <<>> DiG 9.7.0-P1 <<>> gmail.com any
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 30820
;; flags: qr rd ra; QUERY: 1, ANSWER: 4, AUTHORITY: 4, ADDITIONAL: 4

;; QUESTION SECTION:
;gmail.com. IN ANY

;; ANSWER SECTION:
gmail.com. 245972 IN NS ns1.google.com.
gmail.com. 245972 IN NS ns3.google.com.
gmail.com. 245972 IN NS ns4.google.com.
gmail.com. 245972 IN NS ns2.google.com.

;; AUTHORITY SECTION:
gmail.com. 245972 IN NS ns3.google.com.
gmail.com. 245972 IN NS ns1.google.com.
gmail.com. 245972 IN NS ns4.google.com.
gmail.com. 245972 IN NS ns2.google.com.

;; ADDITIONAL SECTION:
ns1.google.com. 308382 IN A 216.239.32.10
ns2.google.com. 308383 IN A 216.239.34.10
ns3.google.com. 308383 IN A 216.239.36.10
ns4.google.com. 308383 IN A 216.239.38.10

;; Query time: 22 msec
;; SERVER: 192.168.1.3#53(192.168.1.3)
;; WHEN: Tue Jan 3 14:38:50 2012
;; MSG SIZE rcvd: 226

.................................................. .................................................. ...
root@mail:/home/optiplex# nslookup gmail.com
Server: 192.168.1.3
Address: 192.168.1.3#53

Non-authoritative answer:
Name: gmail.com
Address: 74.125.236.85
Name: gmail.com
Address: 74.125.236.86
Name: gmail.com
Address: 74.125.236.87

.................................................. .................................................. ....
root@mail:/home/optiplex# nslookup gmail.com
Server: 192.168.1.3
Address: 192.168.1.3#53

Non-authoritative answer:
Name: gmail.com
Address: 74.125.236.85
Name: gmail.com
Address: 74.125.236.86
Name: gmail.com
Address: 74.125.236.87

.................................................. .................................................. ....
root@mail:/home/optiplex# ping gmail.com
PING gmail.com (74.125.236.87) 56(84) bytes of data.

.................................................. .................................................. ....
root@mail:/home/optiplex# cat /etc/resolv.conf
nameserver 192.168.1.3

.................................................. .................................................. ....
root@mail:/home/optiplex# cat /etc/bind/named.conf.options
options {
directory "/var/cache/bind";
query-source address * port 53;

// If there is a firewall between you and nameservers you want
// to talk to, you may need to fix the firewall to allow multiple
// ports to talk. See US-CERT Vulnerability Note VU#800113 - Multiple DNS implementations vulnerable to cache poisoning

// If your ISP provided one or more IP addresses for stable
// nameservers, you probably want to use them as forwarders.
// Uncomment the following block, and insert the addresses replacing
// the all-0's placeholder.

forwarders {
196.200.16.2; 196.200.16.27;

};

auth-nxdomain no; # conform to RFC1035
listen-on-v6 { any; };
};


.................................................. .................................................. ....
root@mail:/home/optiplex# cat /etc/bind/named.conf.local
//
// Do any local configuration here

zone "mosby.com" {
type master;
file "/etc/bind/db.mosby.com";
};

zone "1.168.192.in-addr.arpa" {
type master;
file "/etc/bind/db.1.168.192";
};
//

// Consider adding the 1918 zones here, if they are not used in your
// organization
//include "/etc/bind/zones.rfc1918";

.................................................. .................................................. ....
root@mail:/home/optiplex# cat /etc/bind/db.mosby.com
;
; BIND data file for mosby.com
;
$TTL 604800
@ IN SOA mail.mosby.com. admin.mosby.com. (
070729 ; Serial
604800 ; Refresh
86400 ; Retry
2419200 ; Expire
604800 ) ; Negative Cache TTL
;
@ IN NS mail
IN MX 10 mail
IN A 192.168.1.3
mail IN A 192.168.1.3

.................................................. .................................................. ....
root@mail:/home/optiplex# cat /etc/bind/db.1.168.192

$TTL 3D
@ IN SOA mail.mosby.com. admin.mosby.com. (
3 ; Serial
8H ; Refresh
2H ; Retry
4W ; Expire
1D) ; minimum TTL
NS mail.mosby.com.
10 PTR localhost.

.................................................. .................................................. ....
root@mail:/home/optiplex# cat /etc/hosts
127.0.0.1 localhost.localdomain localhost
192.168.1.3 mail.mosby.com mail

.................................................. .................................................. ...


root@mail:/home/optiplex# tail -f /var/log/zimbra.log
Jan 3 14:51:06 mail postfix/qmgr[3244]: 43A3581ABC: removed
Jan 3 14:51:07 mail amavis[2785]: (02785-01) extra modules loaded: /opt/zimbra/zimbramon/lib/x86_64-linux-gnu-thread-multi/auto/Net/SSLeay/autosplit.ix, /opt/zimbra/zimbramon/lib/x86_64-linux-gnu-thread-multi/auto/Net/SSLeay/randomize.al, IO/Socket/SSL.pm, Net/LDAP/Extension.pm, Net/SSLeay.pm
Jan 3 14:51:10 mail postfix/lmtp[18119]: 97F6C81AB7: to=<admin@mosby.com>, relay=mail.mosby.com[192.168.1.3]:7025, delay=5.5, delays=0.08/0.09/3.5/1.8, dsn=2.1.5, status=sent (250 2.1.5 Delivery OK)
Jan 3 14:51:10 mail postfix/qmgr[3244]: 97F6C81AB7: removed
Jan 3 14:51:46 mail zmmailboxdmgr[18198]: status requested
Jan 3 14:51:46 mail zmmailboxdmgr[18198]: status OK
Jan 3 14:51:46 mail zmmailboxdmgr[18206]: status requested
Jan 3 14:51:46 mail zmmailboxdmgr[18206]: status OK
Jan 3 14:52:07 mail zmmailboxdmgr[18576]: status requested
Jan 3 14:52:07 mail zmmailboxdmgr[18576]: status OK
Jan 3 14:52:48 mail zmmailboxdmgr[18759]: status requested
Jan 3 14:52:48 mail zmmailboxdmgr[18759]: status OK
Jan 3 14:52:48 mail zmmailboxdmgr[18767]: status requested
Jan 3 14:52:48 mail zmmailboxdmgr[18767]: status OK
Jan 3 14:53:24 mail postfix/qmgr[3244]: B18AA81A1E: from=<admin@mosby.com>, size=1103, nrcpt=1 (queue active)
Jan 3 14:53:52 mail zmmailboxdmgr[19042]: status requested
Jan 3 14:53:52 mail zmmailboxdmgr[19042]: status OK
Jan 3 14:53:52 mail zmmailboxdmgr[19050]: status requested
Jan 3 14:53:52 mail zmmailboxdmgr[19050]: status OK
Jan 3 14:53:53 mail postfix/smtp[18904]: connect to mta6.am0.yahoodns.net[67.195.103.232]:25: Connection timed out
Jan 3 14:53:56 mail sshd[19171]: Accepted publickey for zimbra from 192.168.1.3 port 33086 ssh2
Jan 3 14:53:56 mail sshd[19178]: Accepted publickey for zimbra from 192.168.1.3 port 33087 ssh2
Jan 3 14:54:05 mail sshd[19306]: Received disconnect from 192.168.1.3: 11: Closed due to user request.
Jan 3 14:54:06 mail sshd[19305]: Received disconnect from 192.168.1.3: 11: Closed due to user request.
Jan 3 14:54:13 mail zmmailboxdmgr[19776]: status requested
Jan 3 14:54:13 mail zmmailboxdmgr[19776]: status OK
Jan 3 14:54:14 mail postfix/smtp[18904]: connect to mta5.am0.yahoodns.net[74.6.136.244]:25: Connection timed out
Jan 3 14:54:23 mail sshd[19880]: Accepted publickey for zimbra from 192.168.1.3 port 33098 ssh2
Jan 3 14:54:25 mail postfix/qmgr[3244]: 984B281AA3: from=<jrm@mosby.com>, size=1825, nrcpt=1 (queue active)
Jan 3 14:54:25 mail postfix/qmgr[3244]: 87DEC81AA7: from=<zach@sandpconsult.org>, size=1852, nrcpt=1 (queue active)
Jan 3 14:54:25 mail postfix/qmgr[3244]: 7851981AA2: from=<jrm@mosby.com>, size=1661, nrcpt=1 (queue active)
Jan 3 14:54:25 mail postfix/qmgr[3244]: 7842D81A93: from=<jrm@mosby.com>, size=1989, nrcpt=1 (queue active)
Jan 3 14:54:25 mail postfix/qmgr[3244]: C0B6181AA6: from=<zach@sandpconsult.org>, size=1679, nrcpt=1 (queue active)
Jan 3 14:54:25 mail sshd[19944]: Received disconnect from 192.168.1.3: 11: Closed due to user request.
Jan 3 14:54:35 mail postfix/smtp[18904]: connect to mta7.am0.yahoodns.net[98.139.175.225]:25: Connection timed out
Jan 3 14:54:46 mail postfix/smtp[20022]: connect to gmail-smtp-in.l.google.com[209.85.229.27]:25: Connection timed out
Jan 3 14:54:46 mail postfix/smtp[20025]: connect to gmail-smtp-in.l.google.com[209.85.229.27]:25: Connection timed out
Jan 3 14:54:46 mail postfix/smtp[20026]: connect to gmail-smtp-in.l.google.com[209.85.229.27]:25: Connection timed out
Jan 3 14:54:46 mail postfix/smtp[20027]: connect to gmail-smtp-in.l.google.com[209.85.229.27]:25: Connection timed out
Jan 3 14:54:46 mail postfix/smtp[20028]: connect to gmail-smtp-in.l.google.com[209.85.229.27]:25: Connection timed out
Jan 3 14:54:54 mail zmmailboxdmgr[20106]: status requested
Jan 3 14:54:54 mail zmmailboxdmgr[20106]: status OK
Jan 3 14:54:54 mail zmmailboxdmgr[20114]: status requested
Jan 3 14:54:54 mail zmmailboxdmgr[20114]: status OK
Jan 3 14:54:56 mail postfix/smtp[18904]: connect to mta6.am0.yahoodns.net[74.6.136.65]:25: Connection timed out
Jan 3 14:55:07 mail postfix/smtp[20022]: connect to alt1.gmail-smtp-in.l.google.com[209.85.173.26]:25: Connection timed out
Jan 3 14:55:07 mail postfix/smtp[20025]: connect to alt1.gmail-smtp-in.l.google.com[209.85.173.26]:25: Connection timed out
Jan 3 14:55:07 mail postfix/smtp[20026]: connect to alt1.gmail-smtp-in.l.google.com[209.85.173.26]:25: Connection timed out
Jan 3 14:55:07 mail postfix/smtp[20027]: connect to alt1.gmail-smtp-in.l.google.com[209.85.173.26]:25: Connection timed out
Jan 3 14:55:07 mail postfix/smtp[20028]: connect to alt1.gmail-smtp-in.l.google.com[209.85.173.26]:25: Connection timed out
Jan 3 14:55:17 mail postfix/smtp[18904]: connect to mta7.am0.yahoodns.net[74.6.140.64]:25: Connection timed out
Jan 3 14:55:17 mail postfix/smtp[18904]: B18AA81A1E: to=<zackysoft@yahoo.com>, relay=none, delay=11271, delays=11157/0.03/113/0, dsn=4.4.1, status=deferred (connect to mta7.am0.yahoodns.net[74.6.140.64]:25: Connection timed out)
Jan 3 14:55:28 mail postfix/smtp[20022]: connect to alt2.gmail-smtp-in.l.google.com[74.125.127.26]:25: Connection timed out
Jan 3 14:55:28 mail postfix/smtp[20025]: connect to alt2.gmail-smtp-in.l.google.com[74.125.127.26]:25: Connection timed out
Jan 3 14:55:28 mail postfix/smtp[20026]: connect to alt2.gmail-smtp-in.l.google.com[74.125.127.26]:25: Connection timed out
Jan 3 14:55:28 mail postfix/smtp[20027]: connect to alt2.gmail-smtp-in.l.google.com[74.125.127.26]:25: Connection timed out
Jan 3 14:55:28 mail postfix/smtp[20028]: connect to alt2.gmail-smtp-in.l.google.com[74.125.127.26]:25: Connection timed out
Jan 3 14:55:49 mail postfix/smtp[20022]: connect to alt3.gmail-smtp-in.l.google.com[74.125.81.27]:25: Connection timed out
Jan 3 14:55:49 mail postfix/smtp[20025]: connect to alt3.gmail-smtp-in.l.google.com[74.125.81.27]:25: Connection timed out
Jan 3 14:55:49 mail postfix/smtp[20026]: connect to alt3.gmail-smtp-in.l.google.com[74.125.81.27]:25: Connection timed out
Jan 3 14:55:49 mail postfix/smtp[20027]: connect to alt3.gmail-smtp-in.l.google.com[74.125.81.27]:25: Connection timed out
Jan 3 14:55:49 mail postfix/smtp[20028]: connect to alt3.gmail-smtp-in.l.google.com[74.125.81.27]:25: Connection timed out
Jan 3 14:55:57 mail zmmailboxdmgr[20337]: status requested
Jan 3 14:55:57 mail zmmailboxdmgr[20337]: status OK
Jan 3 14:55:57 mail zmmailboxdmgr[20345]: status requested
Jan 3 14:55:57 mail zmmailboxdmgr[20345]: status OK
Jan 3 14:56:04 mail postfix/smtpd[18118]: timeout after END-OF-MESSAGE from localhost.localdomain[127.0.0.1]
Jan 3 14:56:04 mail postfix/smtpd[18118]: disconnect from localhost.localdomain[127.0.0.1]
Jan 3 14:56:09 mail zmmailboxdmgr[20709]: status requested
Jan 3 14:56:09 mail zmmailboxdmgr[20709]: status OK
Jan 3 14:56:10 mail postfix/smtp[20022]: connect to alt4.gmail-smtp-in.l.google.com[209.85.225.26]:25: Connection timed out
Jan 3 14:56:10 mail postfix/smtp[20025]: connect to alt4.gmail-smtp-in.l.google.com[209.85.225.26]:25: Connection timed out
Jan 3 14:56:10 mail postfix/smtp[20026]: connect to alt4.gmail-smtp-in.l.google.com[209.85.225.26]:25: Connection timed out
Jan 3 14:56:10 mail postfix/smtp[20027]: connect to alt4.gmail-smtp-in.l.google.com[209.85.225.26]:25: Connection timed out
Jan 3 14:56:10 mail postfix/smtp[20028]: connect to alt4.gmail-smtp-in.l.google.com[209.85.225.26]:25: Connection timed out
Jan 3 14:56:10 mail postfix/smtp[20022]: 984B281AA3: to=<mbogeus@gmail.com>, relay=none, delay=72753, delays=72647/0.11/105/0, dsn=4.4.1, status=deferred (connect to alt4.gmail-smtp-in.l.google.com[209.85.225.26]:25: Connection timed out)
Jan 3 14:56:10 mail postfix/smtp[20025]: 87DEC81AA7: to=<mbogeus@gmail.com>, relay=none, delay=72751, delays=72646/0.11/105/0, dsn=4.4.1, status=deferred (connect to alt4.gmail-smtp-in.l.google.com[209.85.225.26]:25: Connection timed out)
Jan 3 14:56:10 mail postfix/smtp[20026]: 7851981AA2: to=<mbogeus@gmail.com>, relay=none, delay=72753, delays=72647/0.12/105/0, dsn=4.4.1, status=deferred (connect to alt4.gmail-smtp-in.l.google.com[209.85.225.26]:25: Connection timed out)
Jan 3 14:56:10 mail postfix/smtp[20027]: 7842D81A93: to=<mbogeus@gmail.com>, relay=none, delay=72753, delays=72647/0.13/105/0, dsn=4.4.1, status=deferred (connect to alt4.gmail-smtp-in.l.google.com[209.85.225.26]:25: Connection timed out)
Jan 3 14:56:10 mail postfix/smtp[20028]: C0B6181AA6: to=<mbogeus@gmail.com>, relay=none, delay=72753, delays=72647/0.14/105/0, dsn=4.4.1, status=deferred (connect to alt4.gmail-smtp-in.l.google.com[209.85.225.26]:25: Connection timed out)

[phoenix]

Disable any firewall and/or AppArmor on your server and flush the mail queues, see what happens with that.

[zacheus]

root@ubuntu:/home/optiplex# nmap 192.168.1.3

Starting Nmap 5.00 ( Nmap - Free Security Scanner For Network Exploration & Security Audits. ) at 2012-01-04 14:24 PST
Interesting ports on 192.168.1.3:
Not shown: 985 closed ports
PORT STATE SERVICE
22/tcp open ssh
25/tcp open smtp
53/tcp open domain
110/tcp open pop3
143/tcp open imap
389/tcp open ldap
443/tcp open https
465/tcp open smtps
587/tcp open submission
993/tcp open imaps
995/tcp open pop3s
5222/tcp open unknown
5269/tcp open unknown
7025/tcp open unknown
7777/tcp open unknown
.................................................. .................................................. ....

root@mail:/home/optiplex# iptables -nvL
Chain INPUT (policy ACCEPT 17429 packets, 4660K bytes)
pkts bytes target prot opt in out source destination

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination

Chain OUTPUT (policy ACCEPT 17786 packets, 4630K bytes)
pkts bytes target prot opt in out source destination
.................................................. .................................................. ....

root@mail:/home/optiplex# apparmor_status
apparmor module is loaded.
4 profiles are loaded.
3 profiles are in enforce mode.
/sbin/dhclient3
/usr/lib/NetworkManager/nm-dhcp-client.action
/usr/lib/connman/scripts/dhclient-script
1 profiles are in complain mode.
/usr/sbin/avahi-daemon
2 processes have profiles defined.
0 processes are in enforce mode :
2 processes are in complain mode.
/usr/sbin/avahi-daemon (724)
/usr/sbin/avahi-daemon (725)
0 processes are unconfined but have a profile defined.

please note the appamor module is disabled from startup services.
.................................................. .................................................. ...

I did as you adviced and the mail queues still get stuck in the deferred queue.

the firewall on my gateway router is disabled as well.

I am for the opinion that maybe my ISP has blocked port 25 for smtp

[vavai]

Originally Posted by zacheus

...
I am for the opinion that maybe my ISP has blocked port 25 for smtp

Test it out by telnet-ing some mail server out there, something like :

telnet vavai.net 25

If your ISP didn't block outgoing port 25 as well, your destination mail server will accept the request with the following response :

telnet vavai.net 25
Trying 174.120.9.9...
Connected to vavai.net.
Escape character is '^]'.
220-terraza.websitewelcome.com ESMTP Exim 4.69 #1 Wed, 04 Jan 2012 17:58:21 -0600 
220-We do not authorize the use of this system to transport unsolicited, 
220 and/or bulk e-mail.