bayes db..

[osiris]

sa-learn --dump magic --dbpath /opt/zimbra/amavisd/.spamassassin/
bayes: bayes db version 0 is not able to be used, aborting! at /usr/lib/perl5/vendor_perl/5.8.8/Mail/SpamAssassin/BayesStore/DBM.pm line 196.
ERROR: Bayes dump returned an error, please re-run with -D for more information

-D gives:
"
[25690] dbg: logger: adding facilities: all
[25690] dbg: logger: logging level is DBG
[25690] dbg: generic: SpamAssassin version 3.1.9
[25690] dbg: config: score set 0 chosen.
[25690] dbg: util: running in taint mode? yes
[25690] dbg: util: taint mode: deleting unsafe environment variables, resetting PATH
[25690] dbg: util: PATH included '/opt/zimbra/bin', keeping
[25690] dbg: util: PATH included '/opt/zimbra/zimbramon', keeping
[25690] dbg: util: PATH included '/opt/zimbra/postfix-2.4.3.4z/sbin', keeping
[25690] dbg: util: PATH included '/opt/zimbra/openldap/bin', keeping
[25690] dbg: util: PATH included '/opt/zimbra/snmp/bin', keeping
[25690] dbg: util: PATH included '/opt/zimbra/sleepycat/bin', keeping
[25690] dbg: util: PATH included '/opt/zimbra/openssl/bin', keeping
[25690] dbg: util: PATH included '/opt/zimbra/java/bin', keeping
[25690] dbg: util: PATH included '/usr/kerberos/bin', keeping
[25690] dbg: util: PATH included '/usr/local/bin', keeping
[25690] dbg: util: PATH included '/bin', keeping
[25690] dbg: util: PATH included '/usr/bin', keeping
[25690] dbg: util: final PATH set to: /opt/zimbra/bin:/opt/zimbra/zimbramon:/opt/zimbra/postfix-2.4.3.4z/sbin:/opt/zimbra/openldap/bin/zimbra/snmp/bin:/opt/zimbra/sleepycat/bin:/opt/zimbra/openssl/bin:/opt/zimbra/java/bin:/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin
[25690] dbg: message: ---- MIME PARSER START ----
[25690] dbg: message: main message type: text/plain
[25690] dbg: message: parsing normal part
[25690] dbg: message: added part, type: text/plain
[25690] dbg: message: ---- MIME PARSER END ----
[25690] dbg: dns: is Net::DNS::Resolver available? yes
[25690] dbg: dns: Net::DNS version: 0.59
[25690] dbg: config: using "/etc/mail/spamassassin" for site rules pre files
[25690] dbg: config: read file /etc/mail/spamassassin/init.pre
[25690] dbg: config: read file /etc/mail/spamassassin/v310.pre
[25690] dbg: config: read file /etc/mail/spamassassin/v312.pre
[25690] dbg: config: using "/usr/share/spamassassin" for sys rules pre files
[25690] dbg: config: using "/usr/share/spamassassin" for default rules dir
[25690] dbg: config: read file /usr/share/spamassassin/10_misc.cf
[25690] dbg: config: read file /usr/share/spamassassin/20_advance_fee.cf
[25690] dbg: config: read file /usr/share/spamassassin/20_anti_ratware.cf
[25690] dbg: config: read file /usr/share/spamassassin/20_body_tests.cf
[25690] dbg: config: read file /usr/share/spamassassin/20_compensate.cf
[25690] dbg: config: read file /usr/share/spamassassin/20_dnsbl_tests.cf
[25690] dbg: config: read file /usr/share/spamassassin/20_drugs.cf
[25690] dbg: config: read file /usr/share/spamassassin/20_fake_helo_tests.cf
[25690] dbg: config: read file /usr/share/spamassassin/20_head_tests.cf
[25690] dbg: config: read file /usr/share/spamassassin/20_html_tests.cf
[25690] dbg: config: read file /usr/share/spamassassin/20_meta_tests.cf
[25690] dbg: config: read file /usr/share/spamassassin/20_net_tests.cf
[25690] dbg: config: read file /usr/share/spamassassin/20_phrases.cf
[25690] dbg: config: read file /usr/share/spamassassin/20_****.cf
[25690] dbg: config: read file /usr/share/spamassassin/20_ratware.cf
[25690] dbg: config: read file /usr/share/spamassassin/20_uri_tests.cf
[25690] dbg: config: read file /usr/share/spamassassin/23_bayes.cf
[25690] dbg: config: read file /usr/share/spamassassin/25_accessdb.cf
[25690] dbg: config: read file /usr/share/spamassassin/25_antivirus.cf
[25690] dbg: config: read file /usr/share/spamassassin/25_body_tests_es.cf
[25690] dbg: config: read file /usr/share/spamassassin/25_body_tests_pl.cf
[25690] dbg: config: read file /usr/share/spamassassin/25_dcc.cf
[25690] dbg: config: read file /usr/share/spamassassin/25_dkim.cf
[25690] dbg: config: read file /usr/share/spamassassin/25_domainkeys.cf
[25690] dbg: config: read file /usr/share/spamassassin/25_hashcash.cf
[25690] dbg: config: read file /usr/share/spamassassin/25_pyzor.cf
[25690] dbg: config: read file /usr/share/spamassassin/25_razor2.cf
[25690] dbg: config: read file /usr/share/spamassassin/25_replace.cf
[25690] dbg: config: read file /usr/share/spamassassin/25_spf.cf
[25690] dbg: config: read file /usr/share/spamassassin/25_textcat.cf
[25690] dbg: config: read file /usr/share/spamassassin/25_uribl.cf
[25690] dbg: config: read file /usr/share/spamassassin/30_text_de.cf
[25690] dbg: config: read file /usr/share/spamassassin/30_text_fr.cf
[25690] dbg: config: read file /usr/share/spamassassin/30_text_it.cf
[25690] dbg: config: read file /usr/share/spamassassin/30_text_nl.cf
[25690] dbg: config: read file /usr/share/spamassassin/30_text_pl.cf
[25690] dbg: config: read file /usr/share/spamassassin/30_text_pt_br.cf
[25690] dbg: config: read file /usr/share/spamassassin/50_scores.cf
[25690] dbg: config: read file /usr/share/spamassassin/60_awl.cf
[25690] dbg: config: read file /usr/share/spamassassin/60_whitelist.cf
[25690] dbg: config: read file /usr/share/spamassassin/60_whitelist_dk.cf
[25690] dbg: config: read file /usr/share/spamassassin/60_whitelist_dkim.cf
[25690] dbg: config: read file /usr/share/spamassassin/60_whitelist_spf.cf
[25690] dbg: config: read file /usr/share/spamassassin/60_whitelist_subject.cf
[25690] dbg: config: using "/etc/mail/spamassassin" for site rules dir
[25690] dbg: config: read file /etc/mail/spamassassin/local.cf
[25690] dbg: plugin: loading Mail::SpamAssassin::Plugin::URIDNSBL from @INC
[25690] dbg: plugin: registered Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0xa96ac1 8)
[25690] dbg: plugin: loading Mail::SpamAssassin::Plugin::Hashcash from @INC
[25690] dbg: plugin: registered Mail::SpamAssassin::Plugin::Hashcash=HASH(0xa9f2b6 8)
[25690] dbg: plugin: loading Mail::SpamAssassin::Plugin::SPF from @INC
[25690] dbg: plugin: registered Mail::SpamAssassin::Plugin::SPF=HASH(0xaa16228)
[25690] dbg: plugin: loading Mail::SpamAssassin::Plugin::Pyzor from @INC
[25690] dbg: pyzor: network tests on, attempting Pyzor
[25690] dbg: plugin: registered Mail::SpamAssassin::Plugin::Pyzor=HASH(0xaa2c9a8)
[25690] dbg: plugin: loading Mail::SpamAssassin::Plugin::Razor2 from @INC
[25690] dbg: razor2: razor2 is not available
[25690] dbg: plugin: registered Mail::SpamAssassin::Plugin::Razor2=HASH(0xa9fc264)
[25690] dbg: plugin: loading Mail::SpamAssassin::Plugin::SpamCop from @INC
[25690] dbg: reporter: network tests on, attempting SpamCop
[25690] dbg: plugin: registered Mail::SpamAssassin::Plugin::SpamCop=HASH(0xaabd21c )
[25690] dbg: plugin: loading Mail::SpamAssassin::Plugin::AWL from @INC
[25690] dbg: plugin: registered Mail::SpamAssassin::Plugin::AWL=HASH(0xac1bb34)
[25690] dbg: plugin: loading Mail::SpamAssassin::Plugin::AutoLearnThreshold from @INC
[25690] dbg: plugin: registered Mail::SpamAssassin::Plugin::AutoLearnThreshold=HAS H(0xac24e60)
[25690] dbg: plugin: loading Mail::SpamAssassin::Plugin::WhiteListSubject from @INC
[25690] dbg: plugin: registered Mail::SpamAssassin::Plugin::WhiteListSubject=HASH( 0xac258d4)
[25690] dbg: plugin: loading Mail::SpamAssassin::Plugin::MIMEHeader from @INC
[25690] dbg: plugin: registered Mail::SpamAssassin::Plugin::MIMEHeader=HASH(0xac34 fe4)
[25690] dbg: plugin: loading Mail::SpamAssassin::Plugin::ReplaceTags from @INC
[25690] dbg: plugin: registered Mail::SpamAssassin::Plugin::ReplaceTags=HASH(0xac4 3158)
[25690] dbg: config: adding redirector regex: /^http:\/\/chkpt\.zdnet\.com\/chkpt\/\w+\/(.*)$/i
[25690] dbg: config: adding redirector regex: /^http:\/\/www(?:\d+)?\.nate\.com\/r\/\w+\/(.*)$/i
[25690] dbg: config: adding redirector regex: /^http:\/\/.+\.gov\/(?:.*\/)?externalLink\.jhtml\?.*url=(.*?)(?:&.*)?$/i
[25690] dbg: config: adding redirector regex: /^http:\/\/redir\.internet\.com\/.+?\/.+?\/(.*)$/i
[25690] dbg: config: adding redirector regex: /^http:\/\/(?:.*?\.)?adtech\.de\/.*(?:;|\|)link=(.*?)(?:;|$)/i
[25690] dbg: config: adding redirector regex: m'^http.*?/redirect\.php\?.*(?<=[?&])goto=(.*?)(?:$|[&#])'i
[25690] dbg: config: adding redirector regex: m'^https?:/*(?:[^/]+\.)?emf\d\.com/r\.cfm.*?&r=(.*)'i
[25690] dbg: config: adding redirector regex: m'/(?:index.php)?\?.*(?<=[?&])URL=(.*?)(?:$|[&#])'i
[25690] dbg: config: adding redirector regex: m'^http:/*(?:\w+\.)?google(?:\.\w{2,3}){1,2}/url\?.*?(?<=[?&])q=(.*?)(?:$|[&#])'i
[25690] dbg: config: adding redirector regex: m'^http:/*(?:\w+\.)?google(?:\.\w{2,3}){1,2}/search\?.*?(?<=[?&])q=[^&]*?(?<=%20|..[=+\se:(.*?)(?:$|%20|[\s+&#])'i
[25690] dbg: config: adding redirector regex: m'^http:/*(?:\w+\.)?google(?:\.\w{2,3}){1,2}/search\?.*?(?<=[?&])q=[^&]*?(?<=%20|..[=+\s"|%22)(.*?)(?:$|%22|["\s+&#])'i
[25690] dbg: config: adding redirector regex: m'^http:/*(?:\w+\.)?google(?:\.\w{2,3}){1,2}/translate\?.*?(?<=[?&])u=(.*?)(?:$|[&#])'i
[25690] dbg: plugin: Mail::SpamAssassin::Plugin::ReplaceTags=HASH(0xac4 3158) implements 'finish_parsing_end'
[25690] dbg: replacetags: replacing tags
[25690] dbg: replacetags: done replacing tags
[25690] dbg: bayes: no dbs present, cannot tie DB R/O: /opt/zimbra/.spamassassin/bayes_toks
[25690] dbg: config: score set 1 chosen.
[25690] dbg: bayes: tie-ing to DB file R/O /opt/zimbra/amavisd/.spamassassin/bayes_toks
[25690] dbg: bayes: tie-ing to DB file R/O /opt/zimbra/amavisd/.spamassassin/bayes_seen
[25690] dbg: bayes: found bayes db version 0
bayes: bayes db version 0 is not able to be used, aborting! at /usr/lib/perl5/vendor_perl/5.8.8/Mail/SpamAssassin/BayesStore/DBM.pm li6.
ERROR: Bayes dump returned an error, please re-run with -D for more information
[zimbra@smtp09 ~]$"


This gave me some output before I ran:
sa-learn -p /opt/zimbra/conf/salocal.cf --dbpath /opt/zimbra/amavisd/.spamassassin/ -L --no-sync --spam -f spam.index --showdots where spam.index was a ls > spam.index as instructed at post: Improving spam filtering

Also, when reading trough that log there is a few things I dont get:
1) [25690] dbg: razor2: razor2 is not available <-- thaught this was default in the package now?

2) [25690] dbg: bayes: no dbs present, cannot tie DB R/O: /opt/zimbra/.spamassassin/bayes_toks <-- first it tries there? Seems like there is some mix of config here,as it should be /opt/zimbra/amavisd/.spamassassin/, right? Using default settings most everywhere, except for /opt/zimbra/conf/spamassassin/local.cf which has been edited to what I posted here: Improve antispam


Any ideas what might have happend and how I can fix it?

[osiris]

Quick update on this issue, after running "/opt/zimbra/bin/zmtrainsa" the database seems to fix it self, tried then to run the latest spam from SPAM Archive , same thing happend again, bayes db version is 0 cant continue.. running "/opt/zimbra/bin/zmtrainsa" fixes the problems once again and seems like the data is recorded, first paste after 1st run with 2008/01 from SPAM Archive , then run "/opt/zimbra/bin/zmtrainsa" to correct the db and 2nd paste is after running trough 2008/02 from SPAM Archive and running "/opt/zimbra/bin/zmtrainsa" after.. So things seems to work, with a little back and forth...

[zimbra@smtp09 ~]$ sa-learn --dump magic --dbpath /opt/zimbra/amavisd/.spamassassin/
0.000 0 3 0 non-token data: bayes db version
0.000 0 48569 0 non-token data: nspam
0.000 0 182 0 non-token data: nham
0.000 0 320884 0 non-token data: ntokens
0.000 0 1201182731 0 non-token data: oldest atime
0.000 0 1202568385 0 non-token data: newest atime
0.000 0 1202568246 0 non-token data: last journal sync atime
0.000 0 1202567874 0 non-token data: last expiry atime
0.000 0 1382400 0 non-token data: last expire atime delta
0.000 0 619588 0 non-token data: last expire reduction count
[zimbra@smtp09 ~]$


[zimbra@smtp09 02]$ sa-learn --dump magic --dbpath /opt/zimbra/amavisd/.spamassassin/
0.000 0 3 0 non-token data: bayes db version
0.000 0 63780 0 non-token data: nspam
0.000 0 182 0 non-token data: nham
0.000 0 574841 0 non-token data: ntokens
0.000 0 1201182731 0 non-token data: oldest atime
0.000 0 1202569223 0 non-token data: newest atime
0.000 0 1202569577 0 non-token data: last journal sync atime
0.000 0 1202567874 0 non-token data: last expiry atime
0.000 0 1382400 0 non-token data: last expire atime delta
0.000 0 619588 0 non-token data: last expire reduction count
[zimbra@smtp09 02]$


So my question shifted from previous post to, why is this happening, can I fix this, as I take its not supposed to be like this?

Question 1) and 2) from previous post remains (a mystery?)..

[jeffreyheinen]

We just noticed that same "bayes db version 0" error as well. I was hoping someone already had an answer so I wouldn't have to research this. It could just be a corrupted bayes database that needs to be rebuilt.

I also noticed that the required level for spam is now 6.6 (which lets a lot more through). I wonder if it's related.

I'll run some commands in debug and see if I see similar results.

[quanah]

Originally Posted by osiris

sa-learn --dump magic --dbpath /opt/zimbra/amavisd/.spamassassin/
bayes: bayes db version 0 is not able to be used, aborting! at /usr/lib/perl5/vendor_perl/5.8.8/Mail/SpamAssassin/BayesStore/DBM.pm line 196.
ERROR: Bayes dump returned an error, please re-run with -D for more information
Any ideas what might have happend and how I can fix it?

Easy -- You aren't using the sa-learn from Zimbra's SpamAssassin (/opt/zimbra/libexec/sa-learn). Use the correct sa-learn tool and your problem should be resolved.

--Quanah

[jeffreyheinen]

Then why did it effect me with the zmtrainsa command?

$ zmtrainsa me@myemailaddress.com spam Junk
20080220081900 Starting spamassassin spam training for me@myemailaddress.com using folder Junk
[] INFO: Total messages processed: 150
netset: cannot include 127.0.0.0/8 as it has already been included
Learned tokens from 150 message(s) (150 message(s) examined)
netset: cannot include 127.0.0.0/8 as it has already been included
bayes: bayes db version 0 is not able to be used, aborting! at /opt/zimbra/zimbramon/lib/Mail/SpamAssassin/BayesStore/DBM.pm line 196.
bayes: synced databases from journal in 0 seconds: 92 unique entries (92 total entries)
20080220081912 Finished spamassassin spam training for me@myemailaddress.com using folder Junk

[quanah]

Originally Posted by jeffreyheinen

Then why did it effect me with the zmtrainsa command?

Your issue is different. But it is clear that the original poster was using the wrong sa-learn command. I'm not clear yet why you are hitting an issue.