Page 1 of 2 12 LastLast
Results 1 to 10 of 14

Thread: Spam storm on my ZCS... ideas welcome!

  1. #1
    Labsy is offline Elite Member
    Join Date
    Nov 2009
    Location
    Ljubljana, Slovenia
    Posts
    268
    Rep Power
    5

    Default Spam storm on my ZCS... ideas welcome!

    Hi,

    today I noticed enormous "Deferred" queue on my ZCS 7.1.4 - there were over 20.000 mails in Deferred queue, which is way above normal. Normally, I find 10-100 mails there, most are recipient errors and expired mail addresses.

    After investigating a bit, I found all these deferred mail to be NDR and refusals from other mail servers, mostly from Yahoo and Hotmail:

    Gigabytes! of such logs:
    Code:
    Jan 25 09:33:19 zimbra postfix/smtp[31036]: 8E6EFC1205B: host mta2.am0.yahoodns.net[98.139.54.60] refused to talk to me: 421 4.7.0 [TS01] Messages from my-zimbra-ip temporarily deferred due to user complaints - 4.16.55.1; see http://postmaster.yahoo.com/421-ts01.html
    Jan 25 09:33:19 zimbra postfix/qmgr[2491]: A9536C124CA: from=<okuvow@yahoo.com>, size=1529, nrcpt=21 (queue active)
    Jan 25 09:33:19 zimbra postfix/smtp[31324]: 33BA9C12062: host mx1.mail.eu.yahoo.com[77.238.177.9] refused to talk to me: 421 4.7.0 [TS01] Messages from my-zimbra-ip temporarily deferred due to user complaints - 4.16.55.1; see http://postmaster.yahoo.com/421-ts01.html
    I tried to pinpoint the SOURCE of these mails, meaning which user is compromised... but here I stuck. How to find DOMAIN (I have 300+ domains on this server) and USER (2000+ users)?
    What to look for in logs?
    Which logs?

    Ideas welcome!

  2. #2
    phoenix is offline Zimbra Consultant & Moderator
    Join Date
    Sep 2005
    Location
    Vannes, France
    Posts
    23,470
    Rep Power
    56

    Default

    Search the forums for details on some techniques for blocking NDR spam, you should also reject unlisted recipients (details in the wiki article on improving the Anti-Spam system). If all your deferred queue is spam then block incoming mail at your firewall and purge the deferred queue after that's done open the firewall again.
    Regards


    Bill


    Acompli: A new adventure for Co-Founder KevinH.

  3. #3
    Labsy is offline Elite Member
    Join Date
    Nov 2009
    Location
    Ljubljana, Slovenia
    Posts
    268
    Rep Power
    5

    Default

    Hi Phoenix,

    thanx for quick reply.

    I successfully applied few hacks from Wiki and forum:
    Code:
    vim /opt/zimbra/conf/zmmta.cf
    POSTCONF smtpd_reject_unlisted_recipient            no
    CHANGED TO:
    POSTCONF smtpd_reject_unlisted_recipient            yes
    Works, OK.

    Code:
    vim /opt/zimbra/conf/postfix_recipient_restrictions.cf
    ADDED:
    reject_unknown_recipient_domain
    reject_unverified_recipient
    Works, OK

    Code:
    vim /opt/zimbra/postfix/conf/master.cf
    bounce    unix  -       -       n       -       0       bounce     
    CHANGED TO:
    bounce    unix  -       -       n       -       0       discard
    Hmmm, this one did not survive zmcontrol restart
    But I think this setting is cruical.

    Any idea how to set it up to survive reboot/restart?

    Beside that, after purging 20.000+ deferred queue, applying above (2 of 3) hacks, rebooting...
    here it is how it looks now - messages per hour:
    Last edited by Labsy; 01-25-2012 at 08:45 AM.

  4. #4
    Labsy is offline Elite Member
    Join Date
    Nov 2009
    Location
    Ljubljana, Slovenia
    Posts
    268
    Rep Power
    5

    Default

    Just to confirm:
    is it possible that above mentioned 1st and 2nd change, which both survive reboot, changed the behavior of my ZCS in the manner, that:
    - BEFORE changes ZCS kept NDRs in its own deferred queue and senders did NOT receive it,
    - while AFTER changes sender receives NDR and Greylist 421 responses?

    Why asking? Because many clients, which were used to send mailings around, now complain about not being able to send mail to some recipients to which they were able before. But I think it is just the fact, that before they did not know that some of recipients were either non-existent or had mailbox full or something like.

    Now, my concern:
    If ZCS is now, after I've done changes, sending out NDRs..well, isnt' that just what I wanted to avoid?
    Last edited by Labsy; 01-27-2012 at 01:54 AM.

  5. #5
    Labsy is offline Elite Member
    Join Date
    Nov 2009
    Location
    Ljubljana, Slovenia
    Posts
    268
    Rep Power
    5

    Default

    I am receiving many complaints from users after those changes were applied:
    - some users say, that they receive GREYLISTING response from some servers (and their mail is then delivered)
    - others say, that when sending to multiple recipients, their mail client (Outlook Express) sends out only part of e-mails, until it finds one non-existent address. Then it stops and nobody knows which mails were sent out sucesfully

    I myself do not have neither 500 recipients to test, neither Win XP with Outlook Express to test...huh, bad luck.

    On the other hand, if I turn those settings as they were, I'll receive thousands of bounces again. If I leave settings, users have problems.
    I am stuck.
    any idea?

  6. #6
    Labsy is offline Elite Member
    Join Date
    Nov 2009
    Location
    Ljubljana, Slovenia
    Posts
    268
    Rep Power
    5

    Default

    Seems like this change adds too strict behavoiur rule to mail server, so end-user receives all errors and warnings if on recipient's side is something wrong.
    User's get confused, so I do NOT recommend doing THIS:
    Code:
    vim /opt/zimbra/conf/postfix_recipient_restrictions.cf
    ADD:
    reject_unknown_recipient_domain    <-- Seems OK
    reject_unverified_recipient        <-- Do NOT add this!

  7. #7
    Labsy is offline Elite Member
    Join Date
    Nov 2009
    Location
    Ljubljana, Slovenia
    Posts
    268
    Rep Power
    5

    Default

    Sorry for almost duplicate threads, but I really need advice on how to stop ZCS from spamming around.
    After last step described above, I get thousands of messages in deferred queue, and who knows how much spam has already passed thru successfully...

    Thousands of such can be found in /var/log/mail.info (except of "zimbra" name, all other IP are real, and none of them is known to me):
    Code:
    Jan 31 09:51:34 zimbra postfix/qmgr[9685]: 75E9920C0F9: from=<hjkjdy@live.com>, size=1604, nrcpt=16 (queue active)
    Jan 31 09:51:37 zimbra postfix/smtp[20034]: 75E9920C0F9: to=<betfred@cellarman.dabsol.co.uk>, relay=cellarman.dabsol.co.uk[109.203.99.180]:25, delay=13089, delays=13086/0.11/2.6/0.11, dsn=4.0.0, status=deferred (host cellarman.dabsol.co.uk[109.203.99.180] said: 451 Temporary local problem - please try later (in reply to RCPT TO command))
    Jan 31 09:52:05 zimbra postfix/smtp[20052]: 75E9920C0F9: to=<dave@mostlikelytofail.com>, relay=none, delay=13117, delays=13086/0.12/31/0, dsn=4.4.1, status=deferred (connect to mostlikelytofail.com[74.220.199.6]:25: Connection timed out)

    Is it just me having problems?
    Or even worse - might it be I have problems because of upgrading to ZCS 7.1.4?

    Until lately I only had few dozens of malformed mail, sent from my legitimate users, in deferred queue.
    But lately, there are thousands of them, with FROM and TO addresses of unknown and foolish addresses.

    Server is NOT open relay. Nothing of config has changed in past years.
    Please, help.
    Last edited by Labsy; 01-31-2012 at 02:31 AM.

  8. #8
    raj's Avatar
    raj
    raj is offline Moderator
    Join Date
    Oct 2005
    Location
    USA, Canada and India
    Posts
    777
    Rep Power
    10

    Default

    looks like you have compromised accounts...which are using SMTP AUTH to relay emails using your servers
    Read the following thread and my replies to find out the compromised accounts

    People spamming via my zimbra server

    Raj
    i2k2 Networks
    Dedicated & Shared Zimbra Hosting Provider

  9. #9
    Labsy is offline Elite Member
    Join Date
    Nov 2009
    Location
    Ljubljana, Slovenia
    Posts
    268
    Rep Power
    5

    Default

    Quote Originally Posted by raj View Post
    looks like you have compromised accounts...which are using SMTP AUTH to relay emails using your servers
    Read the following thread and my replies to find out the compromised accounts...
    Hi Raj,
    thank you for hint.
    I have examined my logs with your search string, and increased it for 1 zero:
    Code:
    tail -n 1000000 /var/log/mail.info | grep "sasl_username=" > /tmp/smtplogins.log
    But I found nothing special:
    - there were about 500 rows found in last 1.000.000 bytes of log, which is quite normal operation (I think)
    - repetitions were not more than 3-5 in a row from the same user, which is also quite normal
    - most active users are those half dozen, which send out mailings to few hundreds of recipients...also quite normal

    QUESTION: does each "sasl_username=" log row equals to 1 mail sent out? Ok, to 1 or more recipients.

    QUESTION 2: I still think I would discover a lot about my problem if I could somehow view full body of deferred mail. Any idea how to display them?

  10. #10
    Labsy is offline Elite Member
    Join Date
    Nov 2009
    Location
    Ljubljana, Slovenia
    Posts
    268
    Rep Power
    5

    Default

    Here is another example, how it looks in /var/log/mail.info

    INTERESTING: 123.456.789.1 in this log is IP of my WEB HOSTING cisco firewall. So most probably one of the users has leak in WEB form (too weak captcha).
    But despite that...well, my WEB HOSTING users are all forced to authenticate...so there should be some relation with one of ZIMBRA accounts.

    BUT HEY...how can I find out, which user is this?
    I checked LOG for "sasl_username", but none of web users were authenticated +/- 1 hour to this log!?
    Who is sending out SPAM?
    How to find the account name?

    Code:
    Jan 31 07:16:53 zimbra postfix/smtpd[10539]: 7B35B20C066: client=cisco-out.myFirewall.com[123.456.789.1]
    Jan 31 07:16:53 zimbra postfix/cleanup[10531]: 7B35B20C066: message-id=<20120131061653.7B35B20C066@zimbra.hostname.com>
    Jan 31 07:16:54 zimbra postfix/qmgr[9685]: 7B35B20C066: from=<isax@mail.com>, size=1117, nrcpt=16 (queue active)
    Jan 31 07:16:54 zimbra postfix/cleanup[10531]: 7B6E920C07C: message-id=<20120131061653.7B35B20C066@zimbra.hostname.com>
    Jan 31 07:16:54 zimbra postfix/smtp[10532]: 7B35B20C066: to=<adehomont@aol.com>, relay=127.0.0.1[127.0.0.1]:10024, delay=1.1, delays=0.55/0/0.01/0.53, dsn=2.0.0, status=sent (250 2.0.0 from MTA([127.0.0.1]:10025): 250 2.0.0 Ok: queued as 7B6E920C07C)
    Jan 31 07:16:54 zimbra postfix/smtp[10532]: 7B35B20C066: to=<adeczyk@clerk.com>, relay=127.0.0.1[127.0.0.1]:10024, delay=1.1, delays=0.55/0/0.01/0.53, dsn=2.0.0, status=sent (250 2.0.0 from MTA([127.0.0.1]:10025): 250 2.0.0 Ok: queued as 7B6E920C07C)
    Jan 31 07:16:54 zimbra postfix/smtp[10532]: 7B35B20C066: to=<adehn@freizeit-kids.de>, relay=127.0.0.1[127.0.0.1]:10024, delay=1.1, delays=0.55/0/0.01/0.53, dsn=2.0.0, status=sent (250 2.0.0 from MTA([127.0.0.1]:10025): 250 2.0.0 Ok: queued as 7B6E920C07C)
    Jan 31 07:16:54 zimbra postfix/smtp[10532]: 7B35B20C066: to=<adefish@fsmail.net>, relay=127.0.0.1[127.0.0.1]:10024, delay=1.1, delays=0.55/0/0.01/0.53, dsn=2.0.0, status=sent (250 2.0.0 from MTA([127.0.0.1]:10025): 250 2.0.0 Ok: queued as 7B6E920C07C)
    Jan 31 07:16:54 zimbra postfix/smtp[10532]: 7B35B20C066: to=<adekweiss@gmx.de>, relay=127.0.0.1[127.0.0.1]:10024, delay=1.1, delays=0.55/0/0.01/0.53, dsn=2.0.0, status=sent (250 2.0.0 from MTA([127.0.0.1]:10025): 250 2.0.0 Ok: queued as 7B6E920C07C)
    Jan 31 07:16:54 zimbra postfix/smtp[10532]: 7B35B20C066: to=<ade.casino@googlemail.com>, relay=127.0.0.1[127.0.0.1]:10024, delay=1.1, delays=0.55/0/0.01/0.53, dsn=2.0.0, status=sent (250 2.0.0 from MTA([127.0.0.1]:10025): 250 2.0.0 Ok: queued as 7B6E920C07C)
    Jan 31 07:16:54 zimbra postfix/smtp[10532]: 7B35B20C066: to=<ade@hotmail.com>, relay=127.0.0.1[127.0.0.1]:10024, delay=1.1, delays=0.55/0/0.01/0.53, dsn=2.0.0, status=sent (250 2.0.0 from MTA([127.0.0.1]:10025): 250 2.0.0 Ok: queued as 7B6E920C07C)
    Jan 31 07:16:54 zimbra postfix/smtp[10532]: 7B35B20C066: to=<adeelnadeem786@hotmail.com>, relay=127.0.0.1[127.0.0.1]:10024, delay=1.1, delays=0.55/0/0.01/0.53, dsn=2.0.0, status=sent (250 2.0.0 from MTA([127.0.0.1]:10025): 250 2.0.0 Ok: queued as 7B6E920C07C)
    Jan 31 07:16:54 zimbra postfix/smtp[10532]: 7B35B20C066: to=<ade5@hotmail.de>, relay=127.0.0.1[127.0.0.1]:10024, delay=1.1, delays=0.55/0/0.01/0.53, dsn=2.0.0, status=sent (250 2.0.0 from MTA([127.0.0.1]:10025): 250 2.0.0 Ok: queued as 7B6E920C07C)
    Jan 31 07:16:54 zimbra postfix/smtp[10532]: 7B35B20C066: to=<adeenko@rambler.ru>, relay=127.0.0.1[127.0.0.1]:10024, delay=1.1, delays=0.55/0/0.01/0.53, dsn=2.0.0, status=sent (250 2.0.0 from MTA([127.0.0.1]:10025): 250 2.0.0 Ok: queued as 7B6E920C07C)
    Jan 31 07:16:54 zimbra postfix/smtp[10532]: 7B35B20C066: to=<adehn@t-online.de>, relay=127.0.0.1[127.0.0.1]:10024, delay=1.1, delays=0.55/0/0.01/0.53, dsn=2.0.0, status=sent (250 2.0.0 from MTA([127.0.0.1]:10025): 250 2.0.0 Ok: queued as 7B6E920C07C)
    Jan 31 07:16:54 zimbra postfix/smtp[10532]: 7B35B20C066: to=<adeilza@tele.ch>, relay=127.0.0.1[127.0.0.1]:10024, delay=1.1, delays=0.55/0/0.01/0.53, dsn=2.0.0, status=sent (250 2.0.0 from MTA([127.0.0.1]:10025): 250 2.0.0 Ok: queued as 7B6E920C07C)
    Jan 31 07:16:54 zimbra postfix/smtp[10532]: 7B35B20C066: to=<addys@web.de>, relay=127.0.0.1[127.0.0.1]:10024, delay=1.1, delays=0.55/0/0.01/0.53, dsn=2.0.0, status=sent (250 2.0.0 from MTA([127.0.0.1]:10025): 250 2.0.0 Ok: queued as 7B6E920C07C)
    Jan 31 07:16:54 zimbra postfix/smtp[10532]: 7B35B20C066: to=<adedeoglu@web.de>, relay=127.0.0.1[127.0.0.1]:10024, delay=1.1, delays=0.55/0/0.01/0.53, dsn=2.0.0, status=sent (250 2.0.0 from MTA([127.0.0.1]:10025): 250 2.0.0 Ok: queued as 7B6E920C07C)
    Jan 31 07:16:54 zimbra postfix/smtp[10532]: 7B35B20C066: to=<adehn@web.de>, relay=127.0.0.1[127.0.0.1]:10024, delay=1.1, delays=0.55/0/0.01/0.53, dsn=2.0.0, status=sent (250 2.0.0 from MTA([127.0.0.1]:10025): 250 2.0.0 Ok: queued as 7B6E920C07C)
    Jan 31 07:16:54 zimbra postfix/smtp[10532]: 7B35B20C066: to=<adedic00@yahoo.de>, relay=127.0.0.1[127.0.0.1]:10024, delay=1.1, delays=0.55/0/0.01/0.53, dsn=2.0.0, status=sent (250 2.0.0 from MTA([127.0.0.1]:10025): 250 2.0.0 Ok: queued as 7B6E920C07C)
    Jan 31 07:16:54 zimbra postfix/qmgr[9685]: 7B35B20C066: removed

Page 1 of 2 12 LastLast

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. Help mail server broadcast spam
    By sh1n_b3 in forum Administrators
    Replies: 0
    Last Post: 01-19-2011, 07:44 PM
  2. Trouble Sending mail - All Messages deferred!
    By SiteDiscovery in forum Administrators
    Replies: 7
    Last Post: 09-03-2009, 04:52 AM
  3. Replies: 3
    Last Post: 03-21-2008, 09:47 AM
  4. Replies: 41
    Last Post: 10-29-2007, 02:36 PM
  5. ZCS 3.2 Beta Available
    By KevinH in forum Announcements
    Replies: 31
    Last Post: 07-07-2006, 03:46 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •