Results 1 to 7 of 7

Thread: Server hacked - Tighten up or redeploy?

  1. #1
    264nm is offline Junior Member
    Join Date
    Mar 2014
    Posts
    5
    Rep Power
    1

    Default Server hacked - Tighten up or redeploy?

    Got a dreaded hosts.allow email and realised the damage has probably already been done. Checked /var/log/secure and saw evidence that somebody had been brute forcing, a clear failure on my behalf as I must have misconfigured my denyhosts but didn't see the issue because the server wasn't meant to go live until this weekend after months of it being put off and now I have a big steaming mess on my hands.

    Just would like some advice from anybody who has had an infiltration before and what steps where done to salvage and tighten up everything. Either that or some advice on how the best way to get my settings and redeploy without transferring anything that may compromise the new server.

    Running zimbra 8 on CentOS 6.

    Any help would mean the world to me.

  2. #2
    phoenix is offline Zimbra Consultant & Moderator
    Join Date
    Sep 2005
    Location
    Vannes, France
    Posts
    23,470
    Rep Power
    56

    Default

    Quote Originally Posted by 264nm View Post
    Got a dreaded hosts.allow email and realised the damage has probably already been done. Checked /var/log/secure and saw evidence that somebody had been brute forcing, a clear failure on my behalf as I must have misconfigured my denyhosts but didn't see the issue because the server wasn't meant to go live until this weekend after months of it being put off and now I have a big steaming mess on my hands.

    Just would like some advice from anybody who has had an infiltration before and what steps where done to salvage and tighten up everything. Either that or some advice on how the best way to get my settings and redeploy without transferring anything that may compromise the new server.
    You haven't actually said what type of hack you've had, do you mean compromised accounts or something else? If it's a compromised account (have you read the forum threads on this topic), do you have strong passwords and a lockout policy enforced? If you mean your server has been compromised what exactly has happened?

    Quote Originally Posted by 264nm View Post
    Running zimbra 8 on CentOS 6.
    ZCS "8" is just the major release number, so we know the exact version you're using you should always give the output of the following command:

    Code:
    zmcontrol -v
    Regards


    Bill


    Acompli: A new adventure for Co-Founder KevinH.

  3. #3
    264nm is offline Junior Member
    Join Date
    Mar 2014
    Posts
    5
    Rep Power
    1

    Default

    P.S. My virtual server is setup with /opt/zimbra mounted on a block storage, so the big question I'd like an answer to is regarding how to ensure the contents of this mount is clean of anything that could possibly compromise a new server if the block was attached to a new instance, malicious scripts or sloppy configuration etc...

    The new server will have much more stringent security measures in place, and it'd be ideal if I could simply and safely attach the storage to a new instance.

    I noticed from the admin mail that some person was really trying to send a malicious attachment to one of the accounts and ClamAV was deferring those emails. Because all the new mailbox accounts were setup with a weak default password until the users first login to change it (as mentioned it hadn't gone live yet), there is strong probability that the attacker had logged into a couple of mail accounts.

    Also any brief tips specific to zimbra on how to lock down the new server as much as possible would also be rad.

    I know I'm a new poster so any people who experience the same dramas down the track are welcome to PM for advice on how I dealt with my woes.

  4. #4
    264nm is offline Junior Member
    Join Date
    Mar 2014
    Posts
    5
    Rep Power
    1

    Default

    Quote Originally Posted by phoenix View Post
    You haven't actually said what type of hack you've had, do you mean compromised accounts or something else? If it's a compromised account (have you read the forum threads on this topic), do you have strong passwords and a lockout policy enforced? If you mean your server has been compromised what exactly has happened?

    ZCS "8" is just the major release number, so we know the exact version you're using you should always give the output of the following command:

    Code:
    zmcontrol -v
    Release 8.0.6_GA_5922.RHEL6_64_20131203103705 RHEL6_64 FOSS edition.

    As mentioned in above post it is likely that accounts have been compromised because I wasn't intending to have the server sitting there ready to go live for so long with the same default mailbox password for every user's mail account besides from admin.

    I have read some threads but I'm keen on the specific situation of using the same /opt/zimbra mount on a new server.

    I do use reasonably strong passwords on the server (letters, numbers, UC + LC & symbols), but as mentioned a default password was set for all mail accounts for people to change on first logins.

    I also fear the worst of root shell compromise because as stated an IP address was added to hosts.allow, which if I'm not mistaken would have to have been added as a result of successful ssh access as opposed to hosts.deny which is where the IP should have gone to if I'd configured my denyhosts lockout policies properly. Also I noticed the hosts.allow file was being changed whilst I was in the process of making sure that attacker wasn't able to connect which indicates a probability of shell access.

    Besides from this stuff I'm not too concerned at figuring out the rest of the fine points of what else has been compromised because I'm keen to simply redeploy a new server before the weekend comes instead. I noticed the hosts.allow email within hours of it happening and the bulk of attacking in /var/log/secure was occuring over the past 48 hours.

    Ultimately netsec isn't my area and I don't want to waste too much energy salvaging a bunch of empty mail accounts since the server isn't live yet.

    Thanks
    Last edited by 264nm; 03-11-2014 at 04:28 AM.

  5. #5
    264nm is offline Junior Member
    Join Date
    Mar 2014
    Posts
    5
    Rep Power
    1

    Default

    Also worth noting that I found a few of similar in the logs from IP's not my own...

    "GET /zimlet/com_zimbra_email_dns/xd.jsp HTTP/1.1" 404 1399 "-" "WWW-Mechanize/1.73" 112
    "GET /res/TemplateMsg.js.zgz?skin=../../../../../../../../../opt/zimbra/conf/localconfig.xml%00 HTTP/1.1" 400 1229 "-" "Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1650.57 Safari/537.36" 260

  6. #6
    czguy is offline Senior Member
    Join Date
    Jan 2014
    Posts
    69
    Rep Power
    1

    Default

    If IP addresses were added to hosts.allow then most likely you have a shell compromise. It would be a good thing to check /etc/passwd for any new entries that may or may not have bash/sh attributed to them.

    In my experience, once a host is compromised I do not trust it any longer. I typically will spin up a new VM/server, sanitize the existing data, and do a migration. Although I've only had one "owned" host in my career. The rule of thumb is to never trust a server once it's been hit.

    Considering the situation you're in I'd probably move forward and build a new server with a stronger password policy for ZCS. Set the initial password to something more complex and look into more complex firewall rules with IPTables and/or an IDS system to help alert you to possible intrusion.

  7. #7
    264nm is offline Junior Member
    Join Date
    Mar 2014
    Posts
    5
    Rep Power
    1

    Default

    Quote Originally Posted by czguy View Post
    If IP addresses were added to hosts.allow then most likely you have a shell compromise. It would be a good thing to check /etc/passwd for any new entries that may or may not have bash/sh attributed to them.

    In my experience, once a host is compromised I do not trust it any longer. I typically will spin up a new VM/server, sanitize the existing data, and do a migration. Although I've only had one "owned" host in my career. The rule of thumb is to never trust a server once it's been hit.

    Considering the situation you're in I'd probably move forward and build a new server with a stronger password policy for ZCS. Set the initial password to something more complex and look into more complex firewall rules with IPTables and/or an IDS system to help alert you to possible intrusion.
    Appreciated czguy - /etc/hosts doesn't seem to show much out of the ordinary but I noticed in the obvious attack signs in /log/secure that nobody had a bunch of activity and it's a decent user to hide under
    Mar 10 19:54:23 mail sshd[17003]: Connection closed by 175.111.84.97
    Mar 10 19:54:23 mail sshd[17001]: Connection closed by 175.111.84.97
    Mar 10 19:54:23 mail sshd[17015]: Connection closed by 175.111.84.97
    Mar 10 19:54:23 mail sshd[17013]: Connection closed by 175.111.84.97
    Mar 10 19:54:23 mail sshd[17004]: Failed password for nobody from 175.111.84.97 port 44356 ssh2
    ..etc etc


    Already setting up the new instance that only allows SSH from my IP with denyhosts and iptables, disabled text passes etc. Making sure I keep this one iron tight. This is the first server I've deployed that's gotten pwnd so definitely a good lesson learnt.

    Do I need to do a migration or can I just attach my /opt/zimbra storage block over to new instance? It's not a huge issue to resetup the mail accounts from a fresh install as I've still got the script I used to create ~200 accounts initially. Regardless can you give a few points in sanitizing the existing data?
    Last edited by 264nm; 03-11-2014 at 08:09 AM.

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. Zimbra server hacked
    By LeoB in forum Administrators
    Replies: 9
    Last Post: 02-03-2014, 06:02 PM
  2. Getting very hacked off with Zimbra
    By Guest in forum Administrators
    Replies: 15
    Last Post: 10-04-2011, 06:33 PM
  3. Zimbra got hacked?
    By cocas in forum Administrators
    Replies: 4
    Last Post: 11-23-2010, 02:08 PM
  4. Zimbra server got hacked, security?
    By violentpurr in forum Administrators
    Replies: 5
    Last Post: 03-28-2008, 12:04 AM
  5. Replies: 12
    Last Post: 11-05-2007, 02:48 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •