Page 8 of 9 FirstFirst ... 6789 LastLast
Results 71 to 80 of 81

Thread: Enabling DSPAM

  1. #71
    ewilen's Avatar
    ewilen is offline Moderator
    Join Date
    Jun 2008
    Location
    Berkeley, CA
    Posts
    1,474
    Rep Power
    9

    Default

    Hey, imx, here's another idea, recently posted in [SOLVED] Change DSPAM value. I haven't tested it. Maybe you can give it a try.

    First I'll give drozzini's patch, then a suggestion of my own.

    Drozzini's patch
    Edit /opt/zimbra/amavisd/sbin/amavisd

    Locate the line:
    Code:
    $spam_score = $dspam_result eq 'Spam' ? 10 : -1;  # fabricated
    and change to:
    Code:
    $spam_score = $dspam_result eq 'Spam' ? 4 : -2;  # fabricated
    Some discussion of this section of code can be found at [AMaViS-user] [Help] Amavis and header returned to spamassassin - Open Source Archive

    My alternative
    Looking at the amavisd-new release notes for version 2.6.3, I see a section on configuring the external scanners listed in @spam_scanners. Basically there's a score_factor argument that is multiplied by the DSPAM score to provide a final value. Right now in amavisd.conf.in, the value is 1. To moderate the scores, change it to a smaller value. E.g., edit /opt/zimbra/conf/amavisd.conf.in and change
    Code:
    %%uncomment LOCAL:amavis_dspam_enabled%%         mail_body_size_limit => 64000, score_factor => 1
    to
    Code:
    %%uncomment LOCAL:amavis_dspam_enabled%%         mail_body_size_limit => 64000, score_factor => .5
    After doing this, restart amavisd with zmamavisdctl restart. What I expect you'll see is that the DSPAM scores will range from -.5 to 5 instead of -1 to 10.

    Finally, note that the newest release of amavisd-new now passes DSPAM scores to spamassassin. See the release notes for amavisd-new 2.7.0. Zimbra currently ships with amavisd-new 2.6.4, though.

  2. #72
    drozzini's Avatar
    drozzini is offline New Member
    Join Date
    May 2010
    Location
    Brazil
    Posts
    3
    Rep Power
    5

    Default

    Hey Elliot, i've not noticed that... thats a better way than change the score in code. Thanks!!!

  3. #73
    ewilen's Avatar
    ewilen is offline Moderator
    Join Date
    Jun 2008
    Location
    Berkeley, CA
    Posts
    1,474
    Rep Power
    9

    Default

    You're welcome, drozzini! Please let us know what happens if you try it. (Maybe post a header to show it in action.)

    Then we can edit Using DSPAM for Spam Filtering - Zimbra :: Wiki with our findings.

  4. #74
    ewilen's Avatar
    ewilen is offline Moderator
    Join Date
    Jun 2008
    Location
    Berkeley, CA
    Posts
    1,474
    Rep Power
    9

    Default

    So I enabled DSPAM a little over a week ago, initially using a score_factor of .1. I can confirm that this means DSPAM will add -.1 or 1 depending the determined spam status, so I've added the info to the wiki along with drozzini's method (which I haven't tested).

    Initially as far as I could tell, nothing was being recognized as spam; later, DSPAM started getting more aggressive. However, it was also generating a lot of false positives. This generally wasn't enough to make a difference in terms of the amavis/SA thresholds, but in a way that could be a problem. Same for DSPAM false negatives whose final score still results in them being marked SPAM or SPAMMY by amavis. The problem: it's not entirely clear whether DSPAM is being retrained on its mistakes in those cases.

    I suppose that I could create some Zimbra filters based on a combination of header fields to catch those cases and forward them to the ham & spam accounts, but I haven't done that so far.

    Instead, I increased DSPAM's score_factor to .2, and more important I've trained it using a couple of spam corpuses. I based my steps on HOWTO train or retrain your DSPAM - DirectAdmin Forums

    1. Download ham and spam corpuses from Index of /publiccorpus
    2. Extract them using bzip2 and tar. Note that some of the corpus files at that link extract into the same name, so you'll want to rename any directories that get created, if you download more than one each of ham & spam. Also, I noticed that each directory includes a file called cmd, which you should probably delete.
    3. Pick one spam directory and one ham directory.
    4. As zimbra, do /opt/zimbra/dspam/bin/dspam_train zimbra /path/to/spam_directory /path/to/ham_directory
    5. If desired, repeat with another pair of corpora.

    I don't think there'd be anything wrong with consolidated all the ham corpora into one directory, and the same for all the spam corpora, and then running dspam_train once. Based on the manpage for dspam_train, it doesn't matter if you have different amounts of ham & spam, although documentation for dspam does say that you want to have a fair amount of each.

    You can also get current dspam stats using /opt/zimbra/dspam/bin/dspam_stats -H. If you use more options with dspam_stats, be sure to do one per hyphen, or it may misinterpret them and create a new user folder. (No harm, just delete that folder, which is buried in the dspam hierarchy if I recall correctly.)

    Anyway, I've been able to compare some mail from a mailing list thread which DSPAM was miscategorizing, and after the training it's now marking that mail "Innocent" so I'm hopeful that it will now be more accurate. If & as my confidence in DSPAM increases, I can also increase the scale_factor. I might even consider turning off SA within amavisd.conf.in and just using DSPAM's bayesian filter. In theory, this should be equivalent to using something like ASSP which only uses statistical scoring. There's an argument to be made (e.g. by the author of DSPAM) that this is more accurate and lower-maintenance than manually-tuned rules as found in SA. (Although it should also be noted that DSPAM was intended to do per-user training and analysis instead of sitewide.)
    Last edited by ewilen; 06-06-2011 at 12:11 PM.

  5. #75
    bofh is offline Elite Member
    Join Date
    May 2010
    Posts
    272
    Rep Power
    5

    Default

    even its older i want to drop a note about dspam

    dspam can be absolute accurate or a complete fail - the config of dspam is whats matters

    adjusting the scores is just covering a sideffect of unproper configuration.

    yes training is important but even more is the correct configuration of the algorythm and tokenizer used

    ive wrote a little about it here but you may wanna google there a lot of wikis and explanations how those maths work

    i personally go with the slowest and most complicated :
    tokenizer sbph
    with a graham burtun and pvalue markov
    imporant in that config is that you use trainingmode TOE
    (train on error) not teft or tum


    one more very important thing is a major bug - zimbra uses a very old dspam very the css cleaningtools dont work at all

    also the dspam_clean works only for sql but zimbra uses dspam by default with a css file


    so you need to recompile at least the css cleaning tool
    (ife recompiled hole latest dspam and use that as an replacement) and do a cleaning by crontab

    this is also very important - without cleaning accuracy drops and files are going to big


    with that done my dspam work 10 times better than sa - ife to adjust now the scores in a way dspam gets priority

  6. #76
    ewilen's Avatar
    ewilen is offline Moderator
    Join Date
    Jun 2008
    Location
    Berkeley, CA
    Posts
    1,474
    Rep Power
    9

    Default

    Note that DSPAM is upgraded to 3.10.1 in ZCS 8.O IronMaiden.

    See Bug 62786 - Upgrade to dspam 3.10.1

    I'm going to add a comment suggesting that the upgrade be backported to Helix.

    Regarding your other suggestions, if there are any changes from what you posted in Bug 49649 - Better Dspam IIntegration it would be good mention them there.

  7. #77
    bofh is offline Elite Member
    Join Date
    May 2010
    Posts
    272
    Rep Power
    5

    Default

    Sadly that Dspam improvement Topic ife opened a while ago seems to be no priority.

    My first Intation of that topic was using dspam on Mysql and userbased with preferences section in the settings for each user individual

    i dont think this is gonna happen

    there would be a need of very sophisticated config options in the admin backend because you dont really want really individual usage but you may want groups

    eiher way the database would be very big - could be easy 500mb per user in worst case (or 500 for total server depends on the settings)

    anyway the way spam is filtered would have to be changed big time so my intentional idea behind will not happen

    using dspam wiht only one user is the second best but could be problematic specially for hosting with different sets of user and different languages


    so im not shure if adding to my topic at bug 49649 is a bright idea


    about the config changes - im also not shure if its a good idea to use those settings as default - admins have to get a bit deeper to decide whats really best for them - sbph can easy kill a server with to much traffic and not enough power



    PS: if you want it in helix simply adapt my code sections to redhat
    it shold take you about 15 min top to have dspam replaced -of course own risk
    Last edited by bofh; 01-10-2012 at 03:13 AM.

  8. #78
    MACscr is offline Special Member
    Join Date
    Jun 2010
    Posts
    126
    Rep Power
    5

    Default

    So how do we correct the dspam score of 10 issue in a way thats not going to be overwritten by upgrades? Im using Zimbra 7.2.1

  9. #79
    bofh is offline Elite Member
    Join Date
    May 2010
    Posts
    272
    Rep Power
    5

    Default

    Quote Originally Posted by MACscr View Post
    So how do we correct the dspam score of 10 issue in a way thats not going to be overwritten by upgrades? Im using Zimbra 7.2.1
    we dont

    as every chnage within the conf directory every change is gonna be overwritten by the next upgrade
    so you have to note all those changes for the next upgrade

    sounds aweful but isnt THAT bad, is only a few files anyway could be worse

  10. #80
    MACscr is offline Special Member
    Join Date
    Jun 2010
    Posts
    126
    Rep Power
    5

    Default

    that bad? Its plain pathetic. I dont know of any other software that makes you redo configuration files on every upgrade. Its not like I am wanting to edit source files. Were talking about configs. =/

Page 8 of 9 FirstFirst ... 6789 LastLast

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. Fedora 8 install
    By ahuman in forum Installation
    Replies: 31
    Last Post: 02-15-2008, 08:37 AM
  2. Zimbra install errorr on FC 4 - Please help
    By gva_1030 in forum Installation
    Replies: 16
    Last Post: 02-14-2008, 04:52 PM
  3. Fedora - latest versions?
    By sternfan in forum Installation
    Replies: 8
    Last Post: 01-09-2008, 05:53 AM
  4. Enabling DSPAM
    By plan9 in forum Administrators
    Replies: 6
    Last Post: 07-18-2007, 10:55 AM
  5. Configuring and using DSPAM
    By JoshuaPrismon in forum Administrators
    Replies: 55
    Last Post: 03-02-2007, 09:08 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •