Results 1 to 3 of 3

Thread: Basic Statistics Help

  1. #1
    quietas is offline Elite Member
    Join Date
    Aug 2007
    Location
    Anchorage, AK
    Posts
    376
    Rep Power
    7

    Default Basic Statistics Help

    I need a way to see who is getting how much mail. The mailbox quota is great for how much storage it is taking up, but I need numbers of total mail, and percent received compared to everyone else.

    I'm trying to discover the users getting the most spam basically.
    Culley
    Mail | Dell 2950III | 2x Quad Core 5420 | 8gb RAM | 6x 146gb SAS RAID 0+1 | Red Hat 5.3 | Zimbra 6.0.10 Network Edition
    Test | VMware ESXi Whitebox | Phenom II Black 3.2ghz | 12gb RAM | 6x 1tb SATA RAID 0+1 | CentOS 5.4 | FOSS, Not in use now

  2. #2
    jrefl5 is offline Advanced Member
    Join Date
    Nov 2007
    Location
    AZ, USA
    Posts
    205
    Rep Power
    7

    Default

    If the "mail activity report" from zimbra does not provide enough info I guess you need to write a script or process the zimbra.log file.
    I have a perl script that reads the file and collects some info I'm interested in. It could be improved upon, or modified to pull out the data but you need to dig into the log file to get the regexp set correctly.

    Sorry its a quick and somewhat dirty script, I predate OOP programming by a bit

    Code:
    #!/usr/bin/perl
    # input is the current zimbra logfile
    # this will read the zimbra logfile and create a summary report of
    # postfix, spamassin, and clamAV events
    # note that the default logrotation for zimbra (/etc/logrotate.d/zimbra in Red Hat clones) needs to have
    # "delaycompress" set or I will not be able to read the logfile if used as part of the "postrotate scripts"
    # this script by JCrawford to scratch an itch.
    # it is Licensed  as GPL v3 or later  "jrefl5" ( yahoo ) <com> and others, No warrenty as to usability
    # is provided. james
    #
    # format of zimbra log line is
    # Field                         Position
    # Month                         0-2
    # Day                           4-5
    # Time (HH:MM:SS)   7-14
    # Server name           15-(14+length(server name))
    # Data              end of SNAme+1-> end of line (\n)
    my $CVersion    = "1.0";
    my $Rejects     = 0;
    my $Security    = 0;
    my $Attach              = 0;
    my $SAddress    = 0;
    my $DAddress    = 0;
    my $Discards    = 0;
    my $DSN                 = 0;
    my $Crylic              = 0;
    my $Relay               = 0;
    my $Pipe                = 0;
    my $ConfErr             = 0;
    my $Relay               = 0;
    my $ConfErr             = 0;
    my $ConfErr             = 0;
    my $donknow             = 0;
    my $Blocked             = 0;
    my $BlockBounce = 0;
    my $ClamAV              = 0;
    my $Passed              = 0;
    my $Spammy              = 0;
    my $Clean               = 0;
    my $StDate              = "";
    my $EDate               = "";
    my $inlines             = 0;
    my $currID              = "";
    my $Messages    = 0;
    my $Sender              = "";
    my $Dest                = "";
    my $amavis              = 0;
    my $DefCount    = 0;
    my %Senders             =();
    my %Recivers    =();
    my %MailWare    =();
    my %Deferred    =();
    my %DefMessage  =();
    my %DefRelay    =();
    my %DefTo       =();
    my %DefTime     =();
    # my $emailsenderformat ="^[a-zA-Z][\w\.-]*[a-zA-Z0-9]@[a-zA-Z0-9][\w\.-]*[a-zA-Z0-9]\.[a-zA-Z][a-zA-Z\.]*[a-zA-Z]$";
    InputLoop: while ( <> ) {
            # check beginning of line
            if ($inlines == 0) {
                    $StDate = substr $_, 0, 15;     # Get the start time from the First line in the log
            }
            $inlines+=1;
            $EDate = substr $_, 0, 15;      # Get the time from each line in the log (it may be tha last)
            if ($_ =~ /postfix\/cleanup/gi) {   # start of new message processing
                    $Senders[$Sender]+=1 unless (length($Sender) <= 1);
                    $Recivers[$Dest]+=1 unless (length($Dest) <= 1);
                    if ($_ =~ / ([0-9A-F]{10,}): message-id=/i ) {
                            $currID = $1;       # use this as a hash id for message threads?
                            $Messages+=1;
                            $Sender = "";
                            $Dest = "";
                            # add logic to handle mutilitple messages in process loop at same time
                    }       elsif ($_ =~ /discard/i)        { # track input messages that Postfix doen not queue for amavis
                            $Discards+=1;
                            if ($_ =~ /Crylic/i ) {
                                    $Crylic+=1;
                            }       elsif ($_ =~ /no third-party DSNs/)     {
                                    $DSN+=1;
                            }       else {
                                    print STDOUT "Discard type not Tracked for :\n\t" . $_;
                            }
                    }
            }       elsif ($_ =~ /Passed CLEAN/i)   { # Spamassissin thinks its ok
                    $Passed+=1;
                    $Clean+=1;
            }       elsif ($_ =~ /Passed SPAMMY/i)  { # Spamassissin thinks it may be spam (place in junk mail folder)
                    $Passed+=1;
                    $Spammy+=1;
            }       elsif ($_ =~ /NOQUEUE: reject/i)        { # not placed on queue by postfix bounced back to sending server
                    $Messages+=1;
                    $Rejects+=1;
    #               Use the first form if we choose to track the invalid senders <jhc>
    #               if ($_ =~ /Helo command rejected: need fully-qualified hostname; from=<($emailsenterformat)>/i) {
                    if ($_ =~ /Helo command rejected: need fully-qualified hostname; from=/i)       {
                            $SAddress+=1;
                            # Track invalid senders?
                    }       elsif ($_ =~ /Sender address rejected/i )       {
                            $SAddress+=1;
                            # Track invalid senders?
                    }       elsif ($_ =~ /Recipient address rejected:/i)    { # we don't have that email address localy
                            $DAddress+=1;
                            # Track invalid reciver?
                    }       elsif ($_ =~ /Relay access denied/i)    { # someone wants to relay (ain't no way)
                                    $Relay+=1;
                    }       elsif ($_ =~ /Server configuration error/i)     { # we goofed on a change revert it ASAP
                                    $ConfErr+=1;
                    }       elsif ($_ =~ /Improper use of SMTP command pipelining/i)        { # Sender is not forming SMTP correctly
                                    $Pipe+=1;
                    }       else    {
                            $donknow+=1;
                            print STDOUT "Unknown Reject = \n\t" . $_ ;
                    }
            }       elsif ($_ =~ /Blocked SPAM/)    {
                    $Blocked+=1;
            }       elsif ( $_ =~ /amavis\[\d*\]/i) {
                            if ($_ =~ /Checking:/i) {
                                    $amavis+=1;
                            }
            }       elsif ($_ =~ /clamd\[\d{1,}\]:/i)       { # CalmAV message
    #               Strip out mailware name from log line.
                            if ( $_ =~ /FOUND/ )    {  # Found malware
                                $ClamAV+= 1;
                                # use "'" in regexp instead of "/" due to path nameing in the regexp <jhc>
                                if ($_ =~ m'/opt/zimbra/data/amavisd/tmp/amavis-\d{8,8}T\d{6,6}-(\d{4,5})/parts/\w{1,}:\s{1,}((\w*\.*)*-*\d*)'i)    {
                                    $MailWare{$2}+=1;       # save the malware name and counter for the report
                                }
                            }
            }       elsif ($_ =~ /deferred/g )      {               # its a deferred message we may see the same one many times
                    if ( $_ =~ 'postfix/smtp\[\d{3,5}\]: ([A-F0-9]{12,12}): to=<(\S{1,}?@\S{1,}?.\S{2,5}?)>, relay=([[:print:]]{1,}?)\[\d+\.\d+\.\d+\.\d+\]:25')    {
                            my $msgid = $1;
                            my $address = $2;
                            my $relay = $3;
                            my $err = "I missed the error msg";
                            if ($_ =~ /said: ([[:print:]]{1,})$/) {
                                    $err = $1;
                            } elsif ($_ =~ /talk to me: ([[:print:]]{1,})$/) {
                                    $err = $1;
                            }
                            $Deferred{$msgid}       +=1;
                            $DefMessage{$msgid} = $err;
                            $DefRelay{$msgid}       = $relay;
                            $DefTo{$msgid}          = $address;
                            $DefCount += 1 unless $Deferred{$msgid} > 1;
                            $DefTime{$msgid} = substr $EDate, 7, 8;
                    }
    #       }       else    {
    #               # Unknown Line print it out
    #               print STDOUT "Line not counted\n\t>>>" . $_ . "<<<\n";
            }
    }
    # full file processed now for the report
    print STDOUT "Mail Log Report Ver. " . $CVersion . " for;\n\t" . $StDate . " Through " . $EDate . "\n";
    print STDOUT "\tLog contains             = " . $inlines . " Lines\n";
    print STDOUT "\tTotal Messages processed = " . $Messages . "\n";
    print STDOUT "\tDiscarded messages       = " . $Discards . "\n";
    print STDOUT "\t   Crylic Discards       = " . $Crylic . "\n";
    print STDOUT "\t   BackScatter           = " . $DSN . "\n";
    print STDOUT "\tRejected Messages        = " . $Rejects . "\n";
    print STDOUT "\t   Invalid From address  = " . $SAddress . "\n";
    print STDOUT "\t   Invalid To Address    = " . $DAddress . "\n";
    print STDOUT "\t   Relay Request         = " . $Relay . "\n";
    print STDOUT "\t   Improper Pipelining   = " . $Pipe  . "\n";
    if ($ConfErr > 0)       {
            print STDOUT "\t   Config Error !!! !!!  = " . $ConfErr . "\n";
    }
    if ($donkonw > 0)       {
            print STDOUT "\t   Unknown Rej.          = " . $donknow . "\n";
    }
    print STDOUT " Messages processed by CLAMAV &\n\tSpamAssissin\t\t = " . $amavis . "\n";
    print STDOUT "\tBlocked SPAM  **         = " . $Blocked . "\n";
    print STDOUT "\tPassed Messages          = " . $Passed . "\n";
    print STDOUT "\t     Passed Clean ?      = " . $Clean . "\n";
    print STDOUT "\t     Passed Spammy       = " . $Spammy . "\n";
    print STDOUT "\tClamAV hits **           = " . $ClamAV . "\n";
    if ($ClamAV > 0)        {
            foreach my $kval ( keys %MailWare )     {
                    print STDOUT "\t\t" . $kval . " found " . $MailWare{$kval} . " time";
                    print STDOUT "s" unless ($MailWare{$kval}<2);
                    print STDOUT "\n";
            }
    }
    if ($DefCount > 0)      {               # we had 1 or more deferred emails
            print STDOUT "Unique Deferred messages   = " . $DefCount . "\n";
            foreach my $msgid ( keys %Deferred )    {
                    print STDOUT "\t" . $msgid . " occured " . $Deferred{$msgid} . " times\n\t\taddressed to ";
                    print STDOUT $DefTo{$msgid} . "\n\t\tvia relay " . $DefRelay{$msgid} . "\n\t\tLast responce at ". $DefTime{$msgid} . " was:\n";
                    print STDOUT "\t" . $DefMessage{$msgid} . "\n";
            }
    }
    print STDOUT " ** Note that ClamAV hits are also counted as SPAM\n";

    Some of the numbers from the result don't match with the reported numbers by the Mail Activity report, but they are close, so there is a good chance that some events may be counted more than 1 time.

  3. #3
    quietas is offline Elite Member
    Join Date
    Aug 2007
    Location
    Anchorage, AK
    Posts
    376
    Rep Power
    7

    Default

    I was hoping for something with more options, but that helps.
    Culley
    Mail | Dell 2950III | 2x Quad Core 5420 | 8gb RAM | 6x 146gb SAS RAID 0+1 | Red Hat 5.3 | Zimbra 6.0.10 Network Edition
    Test | VMware ESXi Whitebox | Phenom II Black 3.2ghz | 12gb RAM | 6x 1tb SATA RAID 0+1 | CentOS 5.4 | FOSS, Not in use now

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. Zimbra Disk Statistics, non-EXT3 Volumes
    By ccmacmil in forum Administrators
    Replies: 6
    Last Post: 04-07-2008, 09:59 AM
  2. centos 5 zimbra 4.5.6 no statistics
    By rutman286 in forum Installation
    Replies: 9
    Last Post: 08-14-2007, 09:30 AM
  3. statistics problem
    By rutman286 in forum Installation
    Replies: 5
    Last Post: 08-04-2007, 08:00 AM
  4. Zimbra Basic Client Redirect?
    By kirme3 in forum Administrators
    Replies: 2
    Last Post: 02-24-2007, 11:43 PM
  5. No Status or Statistics Showing
    By tron in forum Administrators
    Replies: 7
    Last Post: 10-17-2005, 09:23 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •