Rewrote zmtrainsa for DSPAM

[unilogic]

I rewrote zmtrainsa to make it a bit more scalable as a training tool for spam filters. My version now grabs the spam or ham messages, and stores them temporarily so more then one spam trainer can take a look at it. Currently I have to split these functions up into two files because of fetchmail's limitations. Right now SA Bayes and DSPAM will be trained. If DSPAM is not installed it will ignore training DSPAM. Also I assumed that dspam was installed in /opt/dspam/bin. If your install is elsewhere change the DSPAMDIR variable in each file. Also, the temp location used is /tmp/sa. That can be changed with the TEMPDIR variable in zmtrainfilter. Run zmtrainsa with its normal usage, zmtrainfilter is really just a support file. Take a look at the files if you need them. The zmtrainsa script now needs zmtrainfilter to run. Although zmtrainfilter doesn't necessarily need zmtrainsa for single file input. I'll comment the code better when I get time. If you have any problems, or don't know why I did something let me know.

-Ben

Download latest fix on the second page of posts.
The files are compressed and attached as zmtrainsa.tgz

[phoenix]

Hi

Unfortunately your scripts don't work for me. the first problem is that zmtrainsa doesn't recognise that zmtrainfilter is in the same directory and I've previously had a problem with zmtrainsa that was resolved by a new script from marcmac in this thread zmtrainsa

Although the new zmtrainsa script was for a problem with spaces in folder names your script exhibits the same problem even if the folder has a single name i.e. Junk.

[unilogic]

Try out this fix, it should fix the above problems. Although I have no idea why it can't filter zmtrainfilter, but I did change that around a bit so hopfully it'll work. The folders with spaces problem carried over from the orignal zmtrainsa that came with M2. Sorry about that. Let me know if it still doesn't work.

Ben

[phoenix]

Tried that fix and it now finds the zmtrainfilter but gives the following error:

$ zmtrainsa server user password spam junk
/opt/zimbra/bin/zmtrainsa: line 70: /opt/zimbra/.fetchmailrc: No such file or directory
chmod: cannot access `/opt/zimbra/.fetchmailrc': No such file or directory
Enter password for bpye@chinook:
fetchmail: lock creation failed.

Syncing SA bayes database

Cleaning up DSpam, this may take a while
dspam_clean starting
PROCESSING USER: zimbra
Processing sigs; age: 14
Signature: 438f67a5116903652140007
    Created: Thu Dec  1 00:00:00 2005

Signature: 438f67a5116911847784999
    Created: Thu Dec  1 00:00:00 2005

...... plus a lot more of the above messages.......

I did enter the correct user password when asked.

[unilogic]

Ah yes, the problem is that its trying to create a file as the zimbra user in the /opt/zimbra folder which zimbra actually has no rights to. So I can either put the .fetchmailrc file in a different place say $TMPDIR or you can chown zimbra:zimbra to /opt/zimbra. Sorry about these mistakes. I've done alot to my install so alot of things aren't at default anymore and I overlook them. I've attached the fixed scripts. Let me know if you have any problems.

-Ben

[phoenix]

Hi

I'm not sure if you wanted me to make any changes for this test but I ran it 'as is' and got these results:

rm: cannot get current directory: Permission denied
mkdir: cannot create directory `/tmp/sa': File exists
fetchmail: lock creation failed.

Syncing SA bayes database

Cleaning up DSpam, this may take a while
dspam_clean starting
PROCESSING USER: zimbra
Processing sigs; age: 14
Signature: 438f67a5116903652140007
    Created: Thu Dec  1 00:00:00 2005

.................. lots more of the above and then this 

Processing probabilities; age: 0
Processing unused; any: 90 quota: 30 nospam: 15 onehit: 15

Cleaning up temp files
rm: cannot get current directory: Permission denied
Done!

Thanks for your effort with this.

[unilogic]

You've run the script as another user then zimbra at some point and its left /tmp/sa. Simply remove that directory manually and all should work. I'll write into the script later today checking that you have permissions for $TMPDIR and throwing an error if you don't. Thanks for being my beta test

-Ben

[phoenix]

I don't think I've run this as anything other than the zimbra user, the /tmp/sa file was still there with zimbra as the owner/group so I deleted it and ran the test again. This is the results I got this time:

fetchmail: lock creation failed.

Syncing SA bayes database
bayes: synced databases from journal in 2 seconds: 237 unique entries (531 total entries)

Cleaning up DSpam, this may take a while
dspam_clean starting
PROCESSING USER: zimbra
Processing sigs; age: 14
Signature: 438f67a5116903652140007
    Created: Thu Dec  1 00:00:00 2005

----- lots more of the above

Signature: 4395997a87243611876707
    Created: Tue Dec  6 00:00:00 2005

Processing probabilities; age: 0
Processing unused; any: 90 quota: 30 nospam: 15 onehit: 15

Cleaning up temp files
rm: cannot get current directory: Permission denied
Done!

The /tmp/sa directory also still exists and the owner/group is zimbra.

[unilogic]

Give this a try. I think what was happening by chance did you su to zimbra, and you ran zmtrainsa. But the current working directory you are still in zimbra doesn't have rights to. I made the scripts change to zimbra's home dir so give it a try. Also a 6th option of where you want the temp directory to be was added. Note: it will run without this option being supplied.

-Ben

[phoenix]

Yes, you're correct I was in a different directory when I ran your zmtrainsa. I wasn't bothering to change as the original zmtrainsa worked. OK, new run and another error I'm afraid:

not found using default location /tmp/sa
fetchmail: lock creation failed.

Syncing SA bayes database

Cleaning up DSpam, this may take a while
dspam_clean starting
PROCESSING USER: zimbra
Processing sigs; age: 14
Signature: 438f67a5116903652140007
    Created: Thu Dec  1 00:00:00 2005
--------------------
Processing probabilities; age: 0
Processing unused; any: 90 quota: 30 nospam: 15 onehit: 15

Cleaning up temp files
Done!