Page 1 of 2 12 LastLast
Results 1 to 10 of 13

Thread: Problems - post 5.0.4 upgrade

  1. #1
    bradb21's Avatar
    bradb21 is offline Advanced Member
    Join Date
    Aug 2007
    Location
    Chicago Area, USA
    Posts
    189
    Rep Power
    7

    Default Problems - post 5.0.4 upgrade

    I've been running 5.0.2 for several months on this current server with no major issues. I upgraded yesterday morning to 5.0.4 and the upgrade was pretty painless (like they've always been). So less than 24 hous after the upgrade I get a call from my wife saying she and her employees can not access their email. I was already at my job, and I don't have remote access to her servers from work, so I said they would have to tough it out until I got out of work. I was getting "internal error" when I tried to access the webmail remotely. I finally talked my wife thought doing a zmcontrol status (all services said running), and then I had her do a zmcontrol stop and start.... this seems to have fixed the problem and everyone can access their mail now.

    So now I'm trying to comb the logs looking for what the heck happened, and I get to my /opt/zimbra/log/mailbox.log (which seems to rotate daily)..... well it's 7.1gb in size on my 10 users server. So I'm willing to bet this will point to my problem, but I don't even know what I can do with a log this size. Any ideas???

    I'm not really wanting this to happen again, as my server was totally stable prior to my 5.0.4 upgrade and before that when I was using OSS (now a paying customer).
    Release 6.0.2_GA_1912.UBUNTU8_64 UBUNTU8_64 NETWORK edition + Mobile Option
    Activesync with Moto Q9C, HTC Touch Pro, Palm Pro, & Palm Pre

  2. #2
    mmorse's Avatar
    mmorse is offline Moderator
    Join Date
    May 2006
    Location
    USA
    Posts
    6,242
    Rep Power
    21

    Default

    Well search the logs for error/fatal/failure/severe, but you might check zmmailboxd.out for crash related material - we need some kinda error to work from.
    ZCS Version: Zimbra 5.0.4 (Network Edition) on Ubuntu 6.06 LTS Server
    I would also apply the fix(s) as in Bug 24897 - update /etc/pam.d/su on ubuntu so ulimits get set properly then note this post: [SOLVED] Too many open files error after increasing zimbra_session_limit_imap

  3. #3
    bradb21's Avatar
    bradb21 is offline Advanced Member
    Join Date
    Aug 2007
    Location
    Chicago Area, USA
    Posts
    189
    Rep Power
    7

    Default

    Quote Originally Posted by mmorse View Post
    Well search the logs for error/fatal/failure/severe, but you might check zmmailboxd.out for crash related material - we need some kinda error to work from.
    I would also apply the fix(s) as in Bug 24897 - update /etc/pam.d/su on ubuntu so ulimits get set properly then note this post: [SOLVED] Too many open files error after increasing zimbra_session_limit_imap
    Yes, this log is just so huge it's tough to work with. I just shutdown Zimbra and I'm trying to gzip up this log so I can move it off the server, and I can play with it more without effecting my server performance. I'll post some as soon as I get this log off the server.

    Thanks!
    Release 6.0.2_GA_1912.UBUNTU8_64 UBUNTU8_64 NETWORK edition + Mobile Option
    Activesync with Moto Q9C, HTC Touch Pro, Palm Pro, & Palm Pre

  4. #4
    bradb21's Avatar
    bradb21 is offline Advanced Member
    Join Date
    Aug 2007
    Location
    Chicago Area, USA
    Posts
    189
    Rep Power
    7

    Default

    This log was really much to large for me to deal with. I had to get my server back up so I did some quick greps for the keywords you gave me and piped to a new log file. I canceled the greps so I could get my server back up. I did get some information, but don't know if it will help. I've gone ahead and made the changes from those two links you provided.

    This is grep for error (you can see by timestamps this is just a stream of errors)
    Code:
    2008-03-31 06:46:57,720 WARN  [ImapSSLServer-35] [name=tracey@domainname.com.com;mid=10;ip=192.168.11.174;] imap - ignoring error during UID FETCH: 
    2008-03-31 06:46:57,723 WARN  [ImapSSLServer-35] [name=tracey@domainname.com.com;mid=10;ip=192.168.11.174;] imap - ignoring error during UID FETCH: 
    2008-03-31 06:46:57,726 WARN  [ImapSSLServer-35] [name=tracey@domainname.com.com;mid=10;ip=192.168.11.174;] imap - ignoring error during UID FETCH: 
    2008-03-31 06:46:57,728 WARN  [ImapSSLServer-35] [name=tracey@domainname.com.com;mid=10;ip=192.168.11.174;] imap - ignoring error during UID FETCH: 
    2008-03-31 06:46:57,731 WARN  [ImapSSLServer-35] [name=tracey@domainname.com.com;mid=10;ip=192.168.11.174;] imap - ignoring error during UID FETCH: 
    2008-03-31 06:46:57,734 WARN  [ImapSSLServer-35] [name=tracey@domainname.com.com;mid=10;ip=192.168.11.174;] imap - ignoring error during UID FETCH: 
    2008-03-31 06:46:57,737 WARN  [ImapSSLServer-35] [name=tracey@domainname.com.com;mid=10;ip=192.168.11.174;] imap - ignoring error during UID FETCH: 
    2008-03-31 06:46:57,740 WARN  [ImapSSLServer-35] [name=tracey@domainname.com.com;mid=10;ip=192.168.11.174;] imap - ignoring error during UID FETCH: 
    2008-03-31 06:46:57,743 WARN  [ImapSSLServer-35] [name=tracey@domainname.com.com;mid=10;ip=192.168.11.174;] imap - ignoring error during UID FETCH: 
    2008-03-31 06:46:57,747 WARN  [ImapSSLServer-35] [name=tracey@domainname.com.com;mid=10;ip=192.168.11.174;] imap - ignoring error during UID FETCH: 
    2008-03-31 06:46:57,752 WARN  [ImapSSLServer-35] [name=tracey@domainname.com.com;mid=10;ip=192.168.11.174;] imap - ignoring error during UID FETCH: 
    2008-03-31 06:46:57,755 WARN  [ImapSSLServer-35] [name=tracey@domainname.com.com;mid=10;ip=192.168.11.174;] imap - ignoring error during UID FETCH: 
    2008-03-31 06:46:57,758 WARN  [ImapSSLServer-35] [name=tracey@domainname.com.com;mid=10;ip=192.168.11.174;] imap - ignoring error during UID FETCH: 
    2008-03-31 06:46:57,760 WARN  [ImapSSLServer-35] [name=tracey@domainname.com.com;mid=10;ip=192.168.11.174;] imap - ignoring error during UID FETCH: 
    2008-03-31 06:46:57,763 WARN  [ImapSSLServer-35] [name=tracey@domainname.com.com;mid=10;ip=192.168.11.174;] imap - ignoring error during UID FETCH: 
    2008-03-31 06:46:57,766 WARN  [ImapSSLServer-35] [name=tracey@domainname.com.com;mid=10;ip=192.168.11.174;] imap - ignoring error during UID FETCH: 
    2008-03-31 06:46:57,769 WARN  [ImapSSLServer-35] [name=tracey@domainname.com.com;mid=10;ip=192.168.11.174;] imap - ignoring error during UID FETCH: 
    2008-03-31 06:46:57,772 WARN  [ImapSSLServer-35] [name=tracey@domainname.com.com;mid=10;ip=192.168.11.174;] imap - ignoring error during UID FETCH: 
    2008-03-31 06:46:57,779 WARN  [ImapSSLServer-35] [name=tracey@domainname.com.com;mid=10;ip=192.168.11.174;] imap - ignoring error during UID FETCH: 
    2008-03-31 06:46:57,782 WARN  [ImapSSLServer-35] [name=tracey@domainname.com.com;mid=10;ip=192.168.11.174;] imap - ignoring error during UID FETCH: 
    2008-03-31 06:46:57,784 WARN  [ImapSSLServer-35] [name=tracey@domainname.com.com;mid=10;ip=192.168.11.174;] imap - ignoring error during UID FETCH:
    This is a sample of grep for failure. (judging from the size of my exports this made up a larger % of log)
    Code:
    com.zimbra.common.service.ServiceException: system failure: IOException while retrieving content for item 1471
    com.zimbra.common.service.ServiceException: system failure: IOException while retrieving content for item 1472
    com.zimbra.common.service.ServiceException: system failure: IOException while retrieving content for item 1473
    com.zimbra.common.service.ServiceException: system failure: IOException while retrieving content for item 1474
    com.zimbra.common.service.ServiceException: system failure: IOException while retrieving content for item 1476
    com.zimbra.common.service.ServiceException: system failure: IOException while retrieving content for item 1477
    com.zimbra.common.service.ServiceException: system failure: IOException while retrieving content for item 1478
    com.zimbra.common.service.ServiceException: system failure: IOException while retrieving content for item 1479
    com.zimbra.common.service.ServiceException: system failure: IOException while retrieving content for item 1480
    com.zimbra.common.service.ServiceException: system failure: IOException while retrieving content for item 1481
    com.zimbra.common.service.ServiceException: system failure: IOException while retrieving content for item 1483
    com.zimbra.common.service.ServiceException: system failure: IOException while retrieving content for item 1484
    com.zimbra.common.service.ServiceException: system failure: IOException while retrieving content for item 1485
    com.zimbra.common.service.ServiceException: system failure: IOException while retrieving content for item 1486
    com.zimbra.common.service.ServiceException: system failure: IOException while retrieving content for item 1487
    com.zimbra.common.service.ServiceException: system failure: IOException while retrieving content for item 1489
    com.zimbra.common.service.ServiceException: system failure: IOException while retrieving content for item 1491
    com.zimbra.common.service.ServiceException: system failure: IOException while retrieving content for item 1492
    com.zimbra.common.service.ServiceException: system failure: IOException while retrieving content for item 1494
    com.zimbra.common.service.ServiceException: system failure: IOException while retrieving content for item 1495
    com.zimbra.common.service.ServiceException: system failure: IOException while retrieving content for item 1496
    com.zimbra.common.service.ServiceException: system failure: IOException while retrieving content for item 1497
    Release 6.0.2_GA_1912.UBUNTU8_64 UBUNTU8_64 NETWORK edition + Mobile Option
    Activesync with Moto Q9C, HTC Touch Pro, Palm Pro, & Palm Pre

  5. #5
    bradb21's Avatar
    bradb21 is offline Advanced Member
    Join Date
    Aug 2007
    Location
    Chicago Area, USA
    Posts
    189
    Rep Power
    7

    Default

    I went and made the changes, but I don't know how to tell if it did anything (basically just uncommented out that line).

    To fix this problem, edit the file: /etc/pam.d/su

    You fill find a line that looks similar to the following:

    # session required pam_limits.so

    You will want to change it so that it reads:

    session required pam_limits.so
    Release 6.0.2_GA_1912.UBUNTU8_64 UBUNTU8_64 NETWORK edition + Mobile Option
    Activesync with Moto Q9C, HTC Touch Pro, Palm Pro, & Palm Pre

  6. #6
    mmorse's Avatar
    mmorse is offline Moderator
    Join Date
    May 2006
    Location
    USA
    Posts
    6,242
    Rep Power
    21

    Default

    su - zimbra
    ulimit -n

    It does seem pretty similar to the first post in that other thread:
    Quote Originally Posted by jeffw View Post
    And here's a sample from the mailbox.log:
    2008-03-25 17:12:03,935 WARN [ImapSSLServer-10] [name=user@domain.org;mid=3;ip=x.x.x.x;] imap - ignoring error during UID FETCH:
    com.zimbra.common.service.ServiceException: system failure: IOException while retrieving content for item 264795
    ExceptionId:ImapSSLServer-10:1206479523935:13358c51319891c4
    Code:service.FAILURE
    at com.zimbra.common.service.ServiceException.FAILURE (ServiceException.java:253)
    at com.zimbra.cs.mailbox.MessageCache.getMimeMessage( MessageCache.java:252)
    at com.zimbra.cs.mailbox.Message.getMimeMessage(Messa ge.java:284)
    at com.zimbra.cs.imap.ImapMessage.getMimeMessage(Imap Message.java:176)
    at com.zimbra.cs.imap.ImapHandler.fetch(ImapHandler.j ava:2937)
    at com.zimbra.cs.imap.ImapHandler.doFETCH(ImapHandler .java:2788)
    at com.zimbra.cs.imap.ImapHandler.executeRequest(Imap Handler.java:402)
    at com.zimbra.cs.imap.TcpImapHandler.processCommand(T cpImapHandler.java:151)
    at com.zimbra.cs.tcpserver.ProtocolHandler.processCon nection(ProtocolHandler.java:160)
    at com.zimbra.cs.tcpserver.ProtocolHandler.run(Protoc olHandler.java:128)
    at EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Wo rker.run(Unknown Source)
    at java.lang.Thread.run(Thread.java:619)
    Caused by: java.io.FileNotFoundException: /opt/zimbra/store/0/3/msg/64/264795-237760.msg (Too many open files)
    at java.io.FileInputStream.open(Native Method)
    at java.io.FileInputStream.<init>(FileInputStream.jav a:106)
    at com.zimbra.cs.store.FileBlobStore.getContent(FileB lobStore.java:371)
    at com.zimbra.cs.store.FileBlobStore.getContent(FileB lobStore.java:365)
    at com.zimbra.cs.mailbox.MessageCache.fetchFromStore( MessageCache.java:270)
    at com.zimbra.cs.mailbox.MessageCache.getMimeMessage( MessageCache.java:214)
    ... 10 more

  7. #7
    mmorse's Avatar
    mmorse is offline Moderator
    Join Date
    May 2006
    Location
    USA
    Posts
    6,242
    Rep Power
    21

    Default

    Any "Too many open files" references?

  8. #8
    bradb21's Avatar
    bradb21 is offline Advanced Member
    Join Date
    Aug 2007
    Location
    Chicago Area, USA
    Posts
    189
    Rep Power
    7

    Default

    [QUOTE=mmorse;85028]su - zimbra
    ulimit -n

    This is my result
    Code:
    root@mailhost:~# su - zimbra
    zimbra@mailhost:~$ ulimit -n
    524288
    zimbra@mailhost:~$
    I was not able to get the log off the server, and had to whack it. I made the change to that file, but it looks like when I do the ulimit... that is probably not the output you were expecting???
    Release 6.0.2_GA_1912.UBUNTU8_64 UBUNTU8_64 NETWORK edition + Mobile Option
    Activesync with Moto Q9C, HTC Touch Pro, Palm Pro, & Palm Pre

  9. #9
    mmorse's Avatar
    mmorse is offline Moderator
    Join Date
    May 2006
    Location
    USA
    Posts
    6,242
    Rep Power
    21

    Default

    That's the correct ulimit (remember you corrected /etc/pam.d/su and/or /etc/pam.d/common-session)

  10. #10
    bradb21's Avatar
    bradb21 is offline Advanced Member
    Join Date
    Aug 2007
    Location
    Chicago Area, USA
    Posts
    189
    Rep Power
    7

    Default

    Just verifying.....

    I sent and changed "/etc/pam.d/su".... by basically uncommented out the line saying: session required pam_limits.so

    used to read:
    #session required pam_limits.so

    now reads:
    session required pam_limits.so
    Release 6.0.2_GA_1912.UBUNTU8_64 UBUNTU8_64 NETWORK edition + Mobile Option
    Activesync with Moto Q9C, HTC Touch Pro, Palm Pro, & Palm Pre

Page 1 of 2 12 LastLast

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. 3.1 Upgrade Problems
    By dee in forum Administrators
    Replies: 22
    Last Post: 07-13-2009, 05:51 PM
  2. Upgrade 5.0.2 to 5.0.4 OSS Failed
    By Grejao in forum Installation
    Replies: 15
    Last Post: 04-04-2008, 01:13 PM
  3. [SOLVED] Upgrade 5.0.2 to 5.0.4 OSS Failed
    By Chewie71 in forum Installation
    Replies: 3
    Last Post: 03-24-2008, 11:01 PM
  4. 4.5.11 to 5.0.2 upgrade completed. No problems.
    By inigoml in forum Installation
    Replies: 1
    Last Post: 02-05-2008, 03:36 PM
  5. Replies: 6
    Last Post: 04-18-2007, 06:03 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •