Time out SSH, SMTP, HTTP

[jessemirza]

Hi Forum,

I have this problem.... we installed ZCS on Debian Etch and it's working fine we have a Shorwall installed.

We are facing a problem that zimbra is not always online it is giving us a timeout i have been monitoring it with Nagios and the strange thing is that i can't find any strange things in the log files of the system neither the zimbra log files.

Does anyone have been facing the same problem?

Please give me some suggestions to examine this problem i am new to ZCS so please be patient and excuse me for my poor English.

Best regards,

Jesse

[phoenix]

Originally Posted by jessemirza

I have this problem.... we installed ZCS on Debian Etch and it's working fine we have a Shorwall installed.

Is this Zimbra Desktop or Zimbra Collaboration Suite (as you've posted in the Desktop forum)?

[jessemirza]

Is there some one willing to help me ? Please ?

[jessemirza]

Hi Forum,

I have this problem.... we installed ZCS on Debian Etch and it's working fine we have a Shorwall installed.

We are facing a problem that zimbra is not always online it is giving us a timeout i have been monitoring it with Nagios and the strange thing is that i can't find any strange things in the log files of the system neither the zimbra log files.

Does anyone have been facing the same problem?

Please give me some suggestions to examine this problem i am new to ZCS so please be patient and excuse me for my poor English.

Best regards,

Jesse

[iway]

Might be a network adapter/networking thing...

[jessemirza]

Hi interways,

Thanks for the fast answer!

I found this in my kernel.log

Apr 29 15:53:18 zimbra kernel: device eth0 entered promiscuous mode
Apr 29 15:53:18 zimbra kernel: audit(1241013198.061:2): dev=eth0 prom=256 old_prom=0 auid=4294967295


Does this explain anything to you ?

Best regards.

Jesse

[iway]

Should be no problem.
Anything in the logs relating to the exact time and date of the problem?

[jessemirza]

Hi There,

Here is the log file to compare with the nagios screen dump

My internal ip is 192.168.1.5 public ip is 80.127.x.x

Thank you in forward

Apr 29 09:58:04 zimbra zimbramon[3157]: 3157:info: 2009-04-29 09:58:01, STATUS: zimbra.mydomain.nl: mailbox: Running
Apr 29 09:58:04 zimbra zimbramon[3157]: 3157:info: 2009-04-29 09:58:01, STATUS: zimbra.mydomain.nl: mta: Running
Apr 29 09:58:04 zimbra zimbramon[3157]: 3157:info: 2009-04-29 09:58:01, STATUS: zimbra.mydomain.nl: snmp: Running
Apr 29 09:58:04 zimbra zimbramon[3157]: 3157:info: 2009-04-29 09:58:01, STATUS: zimbra.mydomain.nl: spell: Running
Apr 29 09:58:04 zimbra zimbramon[3157]: 3157:info: 2009-04-29 09:58:01, STATUS: zimbra.mydomain.nl: stats: Running

Apr 29 09:58:10 zimbra kernel: Shorewall:net2allROP:IN=eth0 OUT= MAC=00:30:48:60:46:9e:00:19:cb:89:e2:f6:08:00 SRC=87.106.7.213 DST=80.127.x.x LEN=48 TOS=0x00 PREC=0x00 TTL=119 ID=25246 DF PROTO=TCP SPT=1183 DPT=1433 WINDOW=65535 RES=0x00 SYN URGP=0

Apr 29 09:58:13 zimbra kernel: Shorewall:net2allROP:IN=eth0 OUT= MAC=00:30:48:60:46:9e:00:19:cb:89:e2:f6:08:00 SRC=87.106.7.213 DST=80.127.x.x LEN=48 TOS=0x00 PREC=0x00 TTL=119 ID=27175 DF PROTO=TCP SPT=1183 DPT=1433 WINDOW=65535 RES=0x00 SYN URGP=0

Apr 29 09:58:29 zimbra kernel: Shorewall:net2allROP:IN=eth0 OUT= MAC=00:30:48:60:46:9e:00:19:cb:89:e2:f6:08:00 SRC=202.170.126.219 DST=80.127.x.x LEN=48 TOS=0x00 PREC=0x00 TTL=111 ID=22358 DF PROTO=TCP SPT=1909 DPT=4899 WINDOW=65535 RES=0x00 SYN URGP=0

Apr 29 09:58:32 zimbra kernel: Shorewall:net2allROP:IN=eth0 OUT= MAC=00:30:48:60:46:9e:00:19:cb:89:e2:f6:08:00 SRC=202.170.126.219 DST=80.127.x.x LEN=48 TOS=0x00 PREC=0x00 TTL=111 ID=24532 DF PROTO=TCP SPT=1909 DPT=4899 WINDOW=65535 RES=0x00 SYN URGP=0

Apr 29 09:59:15 zimbra kernel: Shorewall:all2all:REJECT:IN= OUT=eth1 SRC=80.127.x.x DST=80.69.93.212 LEN=76 TOS=0x00 PREC=0x00 TTL=64 ID=0 DF PROTO=UDP SPT=123 DPT=123 LEN=56
Apr 29 10:00:01 zimbra zimbramon[3829]: 3829:info: 2009-04-29 10:00:01, QUEUE: 0 0

Apr 29 10:00:02 zimbra zimbramon[3836]: 3836:info: 2009-04-29 10:00:01, DISK: zimbra.mydomain.nl: dev: /dev/mapper/vbzimbra-root, mp: /, tot: 148929, avail: 133058
Apr 29 10:00:02 zimbra zimbramon[3836]: 3836:info: 2009-04-29 10:00:01, DISK: zimbra.mydomain.nl: dev: /dev/mapper/vbzimbra-boot, mp: /boot, tot: 198, avail: 169

Apr 29 10:00:04 zimbra zimbramon[3838]: 3838:info: 2009-04-29 10:00:01, STATUS: zimbra.mydomain.nl: antispam: Running

Apr 29 10:00:04 zimbra zimbramon[3838]: 3838:info: 2009-04-29 10:00:01, STATUS: zimbra.mydomain.nl: antivirus: Running

Apr 29 10:00:04 zimbra zimbramon[3838]: 3838:info: 2009-04-29 10:00:01, STATUS: zimbra.mydomain.nl: ldap: Running

Apr 29 10:00:04 zimbra zimbramon[3838]: 3838:info: 2009-04-29 10:00:01, STATUS: zimbra.mydomain.nl: logger: Running

Apr 29 10:00:04 zimbra zimbramon[3838]: 3838:info: 2009-04-29 10:00:01, STATUS: zimbra.mydomain.nl: mailbox: Running

Apr 29 10:00:04 zimbra zimbramon[3838]: 3838:info: 2009-04-29 10:00:01, STATUS: zimbra.mydomain.nl: mta: Running

Apr 29 10:00:04 zimbra zimbramon[3838]: 3838:info: 2009-04-29 10:00:01, STATUS: zimbra.mydomain.nl: snmp: Running
Apr 29 10:00:04 zimbra zimbramon[3838]: 3838:info: 2009-04-29 10:00:01, STATUS: zimbra.mydomain.nl: spell: Running
Apr 29 10:00:04 zimbra zimbramon[3838]: 3838:info: 2009-04-29 10:00:01, STATUS: zimbra.mydomain.nl: stats: Running
Apr 29 10:01:34 zimbra kernel: Shorewall:all2all:REJECT:IN=eth1 OUT= MAC=00:30:48:60:46:9f:00:1f:29:b5:0c:56:08:00 SRC=192.168.1.77 DST=192.168.1.5 LEN=105 TOS=0x00 PREC=0x00 TTL=128 ID=25862 PROTO=UDP SPT=1029 DPT=161 LEN=85
Apr 29 10:01:40 zimbra kernel: Shorewall:all2all:REJECT:IN=eth1 OUT= MAC=00:30:48:60:46:9f:00:1f:29:b5:0c:56:08:00 SRC=192.168.1.77 DST=192.168.1.5 LEN=105 TOS=0x00 PREC=0x00 TTL=128 ID=25867 PROTO=UDP SPT=1029 DPT=161 LEN=85
Apr 29 10:01:46 zimbra kernel: Shorewall:all2all:REJECT:IN=eth1 OUT= MAC=00:30:48:60:46:9f:00:1f:29:b5:0c:56:08:00 SRC=192.168.1.77 DST=192.168.1.5 LEN=105 TOS=0x00 PREC=0x00 TTL=128 ID=25871 PROTO=UDP SPT=1029 DPT=161 LEN=85
Apr 29 10:01:52 zimbra kernel: Shorewall:all2all:REJECT:IN=eth1 OUT= MAC=00:30:48:60:46:9f:00:1f:29:b5:0c:56:08:00 SRC=192.168.1.77 DST=192.168.1.5 LEN=105 TOS=0x00 PREC=0x00 TTL=128 ID=25897 PROTO=UDP SPT=1029 DPT=161 LEN=85

I am smelling a firewall problem but the strange thing is that even with the firewall off the problem does exist !

[samkrieg]

Hi there,

I'm facing the same problem right now, I'll try to give as much valuable informations as I can.

This is a fresh (1 month) installed server.

* OS: CentOS release 5.5 (Final)
* Zimbra: Release 6.0.8_GA_2661.RHEL5_20100820021155 CentOS5 NETWORK edition.
* no iptables installed

Every few minutes the Nagios server goes wild and warns me that some Zimbra services are down.

- Tests with telnet from the client to any zimbra port confirm the issue.
- The problem lasts for like 1 minute and then goes back to normal.
- Only the client suffers from non responsive server.
- Other hosts in the same LAN can access without any problem (unless I try to connect too often).

tcpdump on zimbra server of the problem (port 110)

[root@zimbra log]# tcpdump port 110
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes
11:48:17.775082 IP nagios.domain.tld.58137 > zimbra.domain.tld.pop3: S 2582162921:2582162921(0) win 5840 <mss 1380,sackOK,timestamp 661513095 0,nop,wscale 6>
11:48:20.771473 IP nagios.domain.tld.58137 > zimbra.domain.tld.pop3: S 2582162921:2582162921(0) win 5840 <mss 1380,sackOK,timestamp 661513845 0,nop,wscale 6>
11:48:26.770916 IP nagios.domain.tld.58137 > zimbra.domain.tld.pop3: S 2582162921:2582162921(0) win 5840 <mss 1380,sackOK,timestamp 661515345 0,nop,wscale 6>

This proves that Zimbra is receiving the packets but does not transfer them to the service because nothing changes in file /opt/zimbra/log/mailbox.log

Don't know what's going on here.

I suspect Zimbra to block access from clients if too many connections are made, (or maybe too many opened connection at the same time?)

Some advice is needed. I would like to monitor my server without being waken up at night for nothing :P

Sam

[samkrieg]

OK I think I figured it out.

I edited some sysctl values and it looks like it messes up the network stability.

My advice: do not touch net.ipv4.tcp_tw_reuse or net.ipv4.tcp_tw_recycle as mentionned here:
Performance Tuning Guidelines for Large Deployments - Zimbra :: Wiki (unless you know what you're doing).