Updating Spamassassin In ZCS 7.1.2

[LMStone]

We decided to test upgrading Spamassassin using the sa-update script which Zimbra now includes in /opt/zimbra/zimbramon/bin.

In the event the test was successful; here's what we did:

First, we opened the sa-update script in an editor and found Zimbra had made a number of edits. We wanted to avoid editing the script itself, in effect gambling that Zimbra would not change default paths with subsequent releases.

We saw that the LOCAL_STATE_DIRECTORY mentioned in the script did not exist, so we created it manually as follows:

zimbra@viognier:~> cd data
zimbra@viognier:~/data> ls -al
total 44
drwxr-xr-x 11 zimbra  zimbra 4096 Aug 13 00:06 .
drwxr-xr-x 69 root    root   4096 Aug 13 00:18 ..
drwxr-xr-x  2 zimbra  zimbra 4096 Dec 29  2009 altermime
drwxr-xr-x  9 zimbra  zimbra 4096 Aug 25 22:53 amavisd
drwxr-xr-x  3 zimbra  zimbra 4096 Aug 13 00:06 cbpolicyd
drwxr-xr-x  3 zimbra  zimbra 4096 Apr  6  2008 clamav
drwxr-xr-x  3 zimbra  zimbra 4096 Aug 30 04:08 dspam
drwxr-xr-x  6 zimbra  zimbra 4096 Aug 13 00:03 ldap
drwxr-xr-x  3 zimbra  zimbra 4096 Aug 13 00:05 mailboxd
drwxr-xr-x  4 postfix zimbra 4096 Aug 13 00:06 postfix
drwxrwxrwt 13 zimbra  zimbra 4096 Aug 30 21:46 tmp
zimbra@viognier:~/data> mkdir spamassassin
zimbra@viognier:~/data> chmod 755 spamassassin/
zimbra@viognier:~/data> ls -al
total 48
drwxr-xr-x 12 zimbra  zimbra 4096 Aug 30 21:46 .
drwxr-xr-x 69 root    root   4096 Aug 13 00:18 ..
drwxr-xr-x  2 zimbra  zimbra 4096 Dec 29  2009 altermime
drwxr-xr-x  9 zimbra  zimbra 4096 Aug 25 22:53 amavisd
drwxr-xr-x  3 zimbra  zimbra 4096 Aug 13 00:06 cbpolicyd
drwxr-xr-x  3 zimbra  zimbra 4096 Apr  6  2008 clamav
drwxr-xr-x  3 zimbra  zimbra 4096 Aug 30 04:08 dspam
drwxr-xr-x  6 zimbra  zimbra 4096 Aug 13 00:03 ldap
drwxr-xr-x  3 zimbra  zimbra 4096 Aug 13 00:05 mailboxd
drwxr-xr-x  4 postfix zimbra 4096 Aug 13 00:06 postfix
drwxr-xr-x  2 zimbra  zimbra 4096 Aug 30 21:46 spamassassin
drwxrwxrwt 13 zimbra  zimbra 4096 Aug 30 21:46 tmp

Next, by scrolling through the script you come to a list of the options. We elected to run the update thus:

zimbra@viognier:~/data> cd ../zimbramon/bin/
zimbra@viognier:~/zimbramon/bin> ./sa-update -v --updatedir /opt/zimbra/conf/spamassassin --allowplugins --refreshmirrors
Update available for channel updates.spamassassin.org
Update was available, and was downloaded and installed successfully

There are a number of other channels available, but we wanted to keep Spamassassin fairly vanilla for now, just updated.


So, next we checked that things looked OK:

zimbra@viognier:~/conf/spamassassin> ls -al | grep updates_spamassassin_org
drwxr-x--- 2 zimbra zimbra   4096 Aug 30 21:47 updates_spamassassin_org
-rw-r----- 1 zimbra zimbra   2599 Aug 30 21:47 updates_spamassassin_org.cf
zimbra@viognier:~/conf/spamassassin> cat updates_spamassassin_org.cf 
# UPDATE version 1162027
include updates_spamassassin_org/10_default_prefs.cf
include updates_spamassassin_org/20_advance_fee.cf
include updates_spamassassin_org/20_aux_tlds.cf
include updates_spamassassin_org/20_body_tests.cf
include updates_spamassassin_org/20_compensate.cf
include updates_spamassassin_org/20_dnsbl_tests.cf
include updates_spamassassin_org/20_drugs.cf
include updates_spamassassin_org/20_dynrdns.cf
include updates_spamassassin_org/20_fake_helo_tests.cf
include updates_spamassassin_org/20_freemail.cf
include updates_spamassassin_org/20_freemail_domains.cf
include updates_spamassassin_org/20_head_tests.cf
include updates_spamassassin_org/20_html_tests.cf
include updates_spamassassin_org/20_imageinfo.cf
include updates_spamassassin_org/20_meta_tests.cf
include updates_spamassassin_org/20_net_tests.cf
include updates_spamassassin_org/20_phrases.cf
include updates_spamassassin_org/20_****.cf
include updates_spamassassin_org/20_ratware.cf
include updates_spamassassin_org/20_uri_tests.cf
include updates_spamassassin_org/20_vbounce.cf
include updates_spamassassin_org/23_bayes.cf
include updates_spamassassin_org/25_accessdb.cf
include updates_spamassassin_org/25_antivirus.cf
include updates_spamassassin_org/25_asn.cf
include updates_spamassassin_org/25_dcc.cf
include updates_spamassassin_org/25_dkim.cf
include updates_spamassassin_org/25_hashcash.cf
include updates_spamassassin_org/25_pyzor.cf
include updates_spamassassin_org/25_razor2.cf
include updates_spamassassin_org/25_replace.cf
include updates_spamassassin_org/25_spf.cf
include updates_spamassassin_org/25_textcat.cf
include updates_spamassassin_org/25_uribl.cf
include updates_spamassassin_org/30_text_de.cf
include updates_spamassassin_org/30_text_fr.cf
include updates_spamassassin_org/30_text_it.cf
include updates_spamassassin_org/30_text_nl.cf
include updates_spamassassin_org/30_text_pl.cf
include updates_spamassassin_org/30_text_pt_br.cf
include updates_spamassassin_org/50_scores.cf
include updates_spamassassin_org/60_adsp_override_dkim.cf
include updates_spamassassin_org/60_awl.cf
include updates_spamassassin_org/60_shortcircuit.cf
include updates_spamassassin_org/60_whitelist.cf
include updates_spamassassin_org/60_whitelist_dkim.cf
include updates_spamassassin_org/60_whitelist_spf.cf
include updates_spamassassin_org/60_whitelist_subject.cf
include updates_spamassassin_org/72_active.cf
include updates_spamassassin_org/72_scores.cf
include updates_spamassassin_org/73_sandbox_manual_scores.cf
include updates_spamassassin_org/local.cf
include updates_spamassassin_org/regression_tests.cf
zimbra@viognier:~/conf/spamassassin>

Lastly, while tailing /var/log/zimbra.log in one window, we restarted amavis in another and noted that all the Spamassassin bits were loading correctly.

zimbra@viognier:~/conf/spamassassin> zmamavisdctl restart
Stopping amavisd... done.
Starting amavisd...done.
zimbra@viognier:~/conf/spamassassin>

In the coming weeks we'll create a script we can add to crontab so that this can be automated, but in the interim we thought it would be helpful to post our initial findings here.

Hope that helps, and all suggestions for improving this process are encouraged!

[dlbewley]

Following your procedure worked fine for me on zimbra-core-7.1.2_GA_3268.F13_64-20110804120420.x86_64

I've not enabled it in cron yet.

[thunder04]

Is this also the case for Zimbra 8.0.2? I ran the script and it grabbed updates for us. Before running the script, the folder updates_spammassassin_org and file updates_spamassassin_org.cf did not exist. So I assume I need to schedule this to run myself? If so, how often would one recommend running the sa-update script? Is it a nightly sort of thing? I don't know how often SA rules are modified (if they're like virus definition databases or not...).

[thunder04]

Har har...definitely a RTFM moment.

I found this recommendation on the SpamAssassin site:

How often should I run sa-update?
As often as you like. It typically depends on what time-frame is comfortable for you, and how quickly channels are going to be publishing updates. Generally speaking, once a day is a good starting point.

So, once a day it is!

[thunder04]

Has a script been put together to perform this task? I threw together a basic script that, I thought, would get the job done. However, when the script is ran as zimbra via a cron job, amavisd fails to start. If I run the script by hand as zimbra, it works flawlessly. My only guess is that it's an environment variable/path issue?

Any ideas?

[TomL]

What were the errors that you were getting from your cron job script? I was getting some Perl library errors. My cron script was written in Perl and then calling the Zimbra ones. I resolved my issue with a line "use lib ("/opt/zimbra/zimbramon/lib", "/opt/zimbra/zimbramon/lib/x86_64-linux-thread-multi");" in my Perl script. If you are using bash, try setting the PERLLIB or PERL5LIB environment variable before calling the zimbra scripts.

[TomL]

On the off chance that this will help someone else, here is a Perl script that can be executed nightly via cron as the zimbra user.
It does restart amavisd so there is a small window where mail won't be processed. As with any script that you download off the net, it would be a good idea to verify that it's not going to destroy your Zimbra server before you run it
Tom

[keynet]

On the off chance that this will help someone else, here is a Perl script that can be executed nightly via cron as the zimbra user.

Thanks for your script.
A small note that for those of us still soldiering on with 32 bit systems, you'll need additional environment variables for the 32 bit libs, else amavis won't restart if required

# add 32 bit libs also
use lib ("/opt/zimbra/zimbramon/lib", "/opt/zimbra/zimbramon/lib/x86_64-linux-thread-multi", "/opt/zimbra/zimbramon/lib/i586-linux-thread-multi");

# Clear out any potentially tainted environment variables
delete @ENV{'PATH', 'IFS', 'CDPATH', 'ENV', 'BASH_ENV'};

# Set the environment variables that are needed.
$ENV{'PATH'} = '/bin:/usr/bin';
# add 32 bit libs also
$ENV{'PERLLIB'} = '/opt/zimbra/zimbramon/lib/x86_64-linux-thread-multi:/opt/zimbra/zimbramon/lib:/opt/zimbra/zimbramon/lib/i586-linux-thread-multi';