Results 1 to 9 of 9

Thread: Paralyzing Performance Issues - High CPU/ memory usage

  1. #1
    nitmotwani is offline Junior Member
    Join Date
    Feb 2013
    Posts
    7
    Rep Power
    2

    Default Paralyzing Performance Issues - High CPU/ memory usage

    First things first:
    Release 8.0.2.GA.5569.UBUNTU10.64 UBUNTU10_64 FOSS edition

    Now I know I'm running Zimbra on a less than optimal setup with only 2GB Ram but my setup was working perfectly fine until a few days ago. The web client has been a little slow throughout but over the past couple days it just doesn't open at all. It just keeps loading forever. Yesterday there was the additional issue where mailboxd would keep stopping intermittently. I checked all logs including mailbox.log, zimbra.log etc. but I didn't find any failures or exceptions. The mailboxd issue doesn't exist today but I'm still left with an usable server as the server is completely unresponsive

    The main problem seems to be the CPU utilization which is being consumed totally by a process named zimbravm-cache. Can anyone tell me what this process is and why it's so CPU hungry? Also is it possible that my server is experiencing these issues due to a few new mailboxes that I added recently?

    Please find the output from top below:

    top - 11:59:57 up 282 days, 11:54, 1 user, load average: 8.13, 8.35, 8.45
    Tasks: 91 total, 1 running, 90 sleeping, 0 stopped, 0 zombie
    Cpu(s): 0.2%us, 0.1%sy, 85.7%ni, 6.2%id, 0.0%wa, 0.0%hi, 0.0%si, 7.8%st
    Mem: 2097152k total, 1974012k used, 123140k free, 0k buffers
    Swap: 0k total, 0k used, 0k free, 120560k cached

    PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
    10516 zimbra 20 0 118m 4308 760 S 359 0.2 68003:22 zimbravm-cache
    32222 zimbra 20 0 1208m 451m 15m S 1 22.0 0:53.19 java
    27238 zimbra 20 0 38304 11m 1440 S 0 0.5 0:00.60 perl
    1 root 20 0 23320 416 244 S 0 0.0 0:32.78 init
    2 root 20 0 0 0 0 S 0 0.0 0:00.00 kthreadd/9013
    3 root 20 0 0 0 0 S 0 0.0 0:00.00 khelper/9013


    Thanks for your response in advance

  2. #2
    phoenix is offline Zimbra Consultant & Moderator
    Join Date
    Sep 2005
    Location
    Vannes, France
    Posts
    23,566
    Rep Power
    57

    Default

    Quote Originally Posted by nitmotwani View Post
    First things first:
    Release 8.0.2.GA.5569.UBUNTU10.64 UBUNTU10_64 FOSS edition
    No, the first thing you must do is upgrade your current version of ZCS as a matter of urgency: Urgency on Security Fixes for Bug 80338 and Bug 84547

    Quote Originally Posted by nitmotwani View Post
    Now I know I'm running Zimbra on a less than optimal setup with only 2GB Ram but my setup was working perfectly fine until a few days ago.
    You should upgrade the RAM on this server to at least the minimum requirement specified in the Product Documentation, you will have continuing problems with that amount of RAM.
    Regards


    Bill


    Acompli: A new adventure for Co-Founder KevinH.

  3. #3
    nitmotwani is offline Junior Member
    Join Date
    Feb 2013
    Posts
    7
    Rep Power
    2

    Default

    Thanks for the response Bill. According to my understanding 2GB Ram is considered acceptable according to the system requirements doc here:
    https://www.zimbra.com/docs/os/8.0.0...ml&single=true.

    We've already planned a server upgrade with more juice. As for the upgrade, I'd like to take a full backup before hand and that's a slightly time intensive task.

    I need a fix to this solution asap, however, since it's impacting our users right this moment. Got any pain killers for me here?

  4. #4
    phoenix is offline Zimbra Consultant & Moderator
    Join Date
    Sep 2005
    Location
    Vannes, France
    Posts
    23,566
    Rep Power
    57

    Default

    Quote Originally Posted by nitmotwani View Post
    Thanks for the response Bill. According to my understanding 2GB Ram is considered acceptable according to the system requirements doc here:
    https://www.zimbra.com/docs/os/8.0.0...ml&single=true.
    You'r looking at an old document, the current Product Documentation lists the requirements for the ZCS 8.x versions and you'll always find the current documentation on the page I've given you.

    Quote Originally Posted by nitmotwani View Post
    I need a fix to this solution asap, however, since it's impacting our users right this moment. Got any pain killers for me here?
    Is this actually running in a VM? List the status of the Zimbra services with this command:

    Code:
    zmcontrol status
    Regards


    Bill


    Acompli: A new adventure for Co-Founder KevinH.

  5. #5
    phoenix is offline Zimbra Consultant & Moderator
    Join Date
    Sep 2005
    Location
    Vannes, France
    Posts
    23,566
    Rep Power
    57

    Default

    Sorry, I forgot to add this to my last post. I'd also suggest you take a look at this wiki page https://wiki.zimbra.com/wiki/Investi...curing_Systems (and various forum threads on the topic) and see if you've already been hit by the problems mentioned in my first reply to you - in my opinion that's the most likely problem as the process you list as consuming the CPU cycles isn't a Zimbra process.
    Regards


    Bill


    Acompli: A new adventure for Co-Founder KevinH.

  6. #6
    nitmotwani is offline Junior Member
    Join Date
    Feb 2013
    Posts
    7
    Rep Power
    2

    Default

    Actually I'm pretty sure the problem is related to system requirements and the recent mailboxes that I added. The server did speed up when I deleted a few mailboxes to the extent that the web client did load up albeit fairly slowly.

    I tried running through the doc you mentioned. Didn't detect any vulnerabilities so that confirms my suspicions regarding the systems requirements. I guess migrating to a bigger and better box is the only solution here.

    Thanks for the help again

  7. #7
    phoenix is offline Zimbra Consultant & Moderator
    Join Date
    Sep 2005
    Location
    Vannes, France
    Posts
    23,566
    Rep Power
    57

    Default

    Quote Originally Posted by nitmotwani View Post
    Actually I'm pretty sure the problem is related to system requirements and the recent mailboxes that I added. The server did speed up when I deleted a few mailboxes to the extent that the web client did load up albeit fairly slowly.
    That's one part of the problem.

    Quote Originally Posted by nitmotwani View Post
    I tried running through the doc you mentioned. Didn't detect any vulnerabilities so that confirms my suspicions regarding the systems requirements. I guess migrating to a bigger and better box is the only solution here.
    I'm pretty certain that you've been infected by the 'miner' exploit and it just happens to have a different name than the exploit in the wiki article, as I mention earlier that process has nothing to do with Zimbra. Did you check for any of the additional Zimlets on your server and have you read through the whole document?
    Last edited by phoenix; 07-24-2014 at 10:00 AM.
    Regards


    Bill


    Acompli: A new adventure for Co-Founder KevinH.

  8. #8
    nitmotwani is offline Junior Member
    Join Date
    Feb 2013
    Posts
    7
    Rep Power
    2

    Default

    I manually killed that process without any negative repercussions and the process thankfully stays dead. ATM, the CPU usage is fairly normal but my guess is that's because it's past working hours here and no one is accessing the server.

    I gotta say I'm still a newb and I'm not entirely sure if I'm running through the document correctly. I'm listing out below how I went about the process.

    1. Zimlets recently installed (found by running find /opt/zimbra/zimlets-deployed -mtime -35 -o -ctime -35)
    /opt/zimbra/zimlets-deployed
    /opt/zimbra/zimlets-deployed/com_zimbra_dnd
    /opt/zimbra/zimlets-deployed/com_zimbra_dnd/dnd.jsp

    2. JSP identified as dnd.jsp
    3. Attempted undeploy of zimlet (zmzimletctl undeploy com_zimbra_dnd) causes exception in mailbox.log:
    ExceptionId:qtp711772275-1047:https://xxx.xx.xxx.xx:7071/service/a...e50a21b51ccf1d
    Code:account.NO_SUCH_ZIMLET
    Not sure, how to proceed here. How can I undeploy this zimlet?

    4. find /opt/zimbra -iname \*.jsp -exec grep -i getRuntime {} \; - returns nothing at all
    5. find /var/tmp /tmp -iname miner\* - returns nothing at all
    6. Removed all files from /tmp and /var/tmp folder
    7. Running source ~/bin/zmshutil; zmsetvars; ldapsearch -x -LLL -b "" -H ldap://xxx.xxx.com :389 -D "$zimbra_ldap_userdn" -w "$zimbra_ldap_password" '(zimbraIsAdminAccount=TRUE)' dn | grep cn=admin

    Returned:
    dn: cn=admins,cn=zimbra
    dn: uid=zimbra,cn=admins,cn=zimbra
    dn: uid=zmreplica,cn=admins,cn=zimbra

    Are any of these rogue admin accounts?

    8. grep -B 3 PARSE_ERROR /opt/zimbra/log/mailbox.log | egrep '/service/soap:|/service/admin/soap:' - returned nothing at all


    Would appreciate a bit of guidance to see if I'm going about this the right way

  9. #9
    quanah is offline Zimbra Employee
    Join Date
    May 2007
    Location
    Zimbra
    Posts
    1,276
    Rep Power
    10

    Default

    I would also note that 8.0.2 suffers from spontaneous database corruption, you should upgrade from it ASAP.
    Quanah Gibson-Mount
    Server Architect
    Zimbra, Inc
    --------------------
    Zimbra :: the leader in open source messaging and collaboration

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. High memory usage
    By mpcarroll1 in forum Users
    Replies: 2
    Last Post: 03-01-2012, 05:31 PM
  2. High memory usage
    By mpcarroll1 in forum General Questions
    Replies: 2
    Last Post: 03-01-2012, 05:31 PM
  3. Memory Usage is high??!!!
    By Wilson_Lee in forum Installation
    Replies: 4
    Last Post: 10-02-2008, 09:16 AM
  4. High Swap File Usage, Lots of Memory Free
    By krknopp in forum Administrators
    Replies: 6
    Last Post: 04-22-2008, 12:53 AM
  5. Extremely High Memory Usage for Amavisd
    By atomicfission in forum Installation
    Replies: 4
    Last Post: 10-02-2006, 02:23 PM

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •