AFFECTED ZCS VERSIONS: 8.0.0, 8.0.1, 8.0.2



PROBLEM


The default denial of service filter (DoSFilter) configuration can block valid service requests for mobile sync, sharing, Zimbra Desktop, Zimbra Connector for Outlook and administration tasks. The issue can be identified by a "503 Service Unavailable" error in sync.log for the affected client and a "DoSFilter:DOS ALERT" warning in zmmailboxd.out.

sync.log:
Code:
2013-01-15 15:52:20,426 WARN [qtp1635701107-91:https://10.10.0.54:443/Microsoft-Server-ActiveSync?User=zsupport2&DeviceId=Appl5K0113UN3NR&DeviceType=iPhone&Cmd=FolderSync][name=zsupport2@domain.com;mid=64;ip=71.194.89.54;Cmd=FolderSync;DeviceID=Appl5K0113UN3NR;Version=12.1;] sync - Service exception

com.zimbra.common.service.ServiceException: error while proxying request to target server: HTTP/1.1 503 Service Unavailable 
ExceptionId:qtp1635701107-91:https://10.10.0.54:443/Microsoft-Server-ActiveSync?User=zsupport2&DeviceId=Appl5K0113UN3NR&DeviceType=iPhone&Cmd=FolderSync:1358286740426:c5ca7f36bb0a038f Code:service.PROXY_ERROR Arg:(url, STR,"http://mail.domain.com:80/service/soap/SyncRequest")
zmmailboxd.out:
Code:
2013-01-15 15:57:32.537:WARN:oejs.DoSFilter:DOS ALERT:ip=127.0.1.1,session=null,user=null
This can affect server to server intercommunication which would otherwise be trusted.

IMPACT


End users may be unable to fully sync shared contacts over mobile (ActiveSync), Zimbra Desktop, or Zimbra Connector for Outlook. Administrators may be unable to perform some administrative tasks with zmmailbox.

BUGS


Jetty DoSFiler (DOS ALERT) causing commands piped to zmmailbox to fail
DoSFilter: add LC value for delayMs
Use LDAP attribute for DOS filter whitelist

WORKAROUND


8.0.2 only

1) Remove this block from the /opt/zimbra/jetty/etc/service.web.xml.in configuration template:
Code:
<init-param>
  <param-name>delayMs</param-name>
  <param-value>-1</param-value>
</init-param>
This will disable temporary service shutdown for clients reaching the maximum number of allowed requests per second (default: 30) and invoke a 100ms connection delay when the request threshold is reached.

2) Save the configuration template file.

3) Restart mailboxd:
Code:
zmmailboxdctl restart
Optional for 8.0.2
You may increase the maximum allowed number of service requests per second which raises the request threshold before a client is throttled.

1) Update the local configuration key zimbra_dos_filter_max_requests_per_sec (default: 30):
Code:
zmlocalconfig -e zimbra_dos_filter_max_requests_per_sec=100
2) Restart mailboxd:
Code:
zmmailboxdctl restart
Prior to 8.0.2, this configuration must be modified in the /opt/zimbra/jetty/etc/service.web.xml.in configuration template:
Code:
<init-param>
  <param-name>maxRequestsPerSec</param-name>
  <param-value>30</param-value>
</init-param>
1) Set the desired value.

2) Restart mailboxd:
Code:
zmmailboxdctl restart
8.0.0, 8.0.1, 8.0.2 only
It is possible to whitelist certain IP addresses to avoid triggering DoSFilter. If this is desired, we recommend whitelisting only other ZCS servers in your environment.

1) Edit the /opt/zimbra/jetty/etc/service.web.xml.in configuration template by adding internal ZCS servers to the IP Whitelist.
Code:
<init-param>
  <param-name>ipWhitelist</param-name>
  <param-value>%%zimbraLocalBindAddress%%</param-value>
</init-param>
In param-value, add a comma-separated list of IP addresses for which DoS filtering will be bypassed. Be sure to preserve the %%zimbraLocalBindAddress%% substitution variable. For example, for ZCS servers having IP addresses 10.1.2.3, 10.1.2.4 and 10.1.2.5:
Code:
<init-param>
  <param-name>ipWhitelist</param-name>
  <param-value>%%zimbraLocalBindAddress%%,10.1.2.3,10.1.2.4,10.1.2.5</param-value>
</init-param>
2) Save the configuration template file.

3) Restart mailboxd:
Code:
zmmailboxdctl restart
FIX TARGETED


ZCS 8.0.3 will provide a mechanism for setting and preserving DoSFilter settings going forward. Upgrades to versions prior to 8.0.3 will require the manual changes stated above.