LDAP Problems

[brained]

OK, ran into a little problem - Sever is Centos 4.4 running zcs-4.0.4_GA_457.RHEL4.

Installation went fine but the hardware was weak so I brought up new hardware with a RAID10 and dualcore to help things out.
Installed zcs on the new server.
Shutdown zcs services on new server.
Shutdown zcs services on old server.
Ran an rsync from the old server to the backup location (I do this daily anyways)
Ran an rsync from the backup location to the new server.
Duplicated the IP addresses and host info from the old server to the new server.
Reran the installation process and allowed it to upgrade on the new server.
Started services on the new server.
Bingo! Everything works, or so I thought. Everything user related works fine.
However, I can't create anything new in the admin area.

Attempting to create a new user results in the error at the bottom. Adding aliases, new domains, etc results in similar errors. zmprov also errors out although not so verbose.

Shutdown the services and ran zmfixperms - no difference.
Shutdown the services and re-ran the installation - got this the first time:

The system will be modified. Continue? [N] y

Shutting down zimbra mail

Backing up ldap

bdb(): DB_ENV->log_flush: LSN of 1/3692191 past current end-of-log of 1/3641691
bdb(): Database environment corrupt; the wrong log files may have been removed or incompatible database files imported from another environment

The last part repeated with variations for a few hundred lines.

And then the installation completed. No difference - Everything user related works fine (new mails, new documents, new folders) - anything related to creating new things in admin gives the error at the bottom.

Message: system failure: unable to create alias: [LDAP: error code 80 - index generation failed]
com.zimbra.cs.service.ServiceException: system failure: unable to create alias: [LDAP: error code 80 - index generation failed]
at com.zimbra.cs.service.ServiceException.FAILURE(Ser viceException.java:174)
at com.zimbra.cs.account.ldap.LdapProvisioning.addAli asInternal(LdapProvisioning.java:1019)
at com.zimbra.cs.account.ldap.LdapProvisioning.addAli as(LdapProvisioning.java:970)
at com.zimbra.cs.service.admin.AddAccountAlias.handle (AddAccountAlias.java:75)
at com.zimbra.soap.SoapEngine.dispatchRequest(SoapEng ine.java:261)
at com.zimbra.soap.SoapEngine.dispatch(SoapEngine.jav a:162)
at com.zimbra.soap.SoapEngine.dispatch(SoapEngine.jav a:84)
at com.zimbra.soap.SoapServlet.doPost(SoapServlet.jav a:223)
at javax.servlet.http.HttpServlet.service(HttpServlet .java:709)
at com.zimbra.cs.servlet.ZimbraServlet.service(Zimbra Servlet.java:173)
at javax.servlet.http.HttpServlet.service(HttpServlet .java:802)
at org.apache.catalina.core.ApplicationFilterChain.in ternalDoFilter(ApplicationFilterChain.java:252)
at org.apache.catalina.core.ApplicationFilterChain.do Filter(ApplicationFilterChain.java:173)
at org.apache.catalina.core.StandardWrapperValve.invo ke(StandardWrapperValve.java:213)
at org.apache.catalina.core.StandardContextValve.invo ke(StandardContextValve.java:178)
at org.apache.catalina.core.StandardHostValve.invoke( StandardHostValve.java:126)
at org.apache.catalina.valves.ErrorReportValve.invoke (ErrorReportValve.java:105)
at org.apache.catalina.core.StandardEngineValve.invok e(StandardEngineValve.java:107)
at org.apache.catalina.valves.AccessLogValve.invoke(A ccessLogValve.java:541)
at org.apache.catalina.connector.CoyoteAdapter.servic e(CoyoteAdapter.java:148)
at org.apache.coyote.http11.Http11Processor.process(H ttp11Processor.java:869)
at org.apache.coyote.http11.Http11BaseProtocol$Http11 ConnectionHandler.processConnection(Http11BaseProt ocol.java:667)
at org.apache.tomcat.util.net.PoolTcpEndpoint.process Socket(PoolTcpEndpoint.java:527)
at org.apache.tomcat.util.net.LeaderFollowerWorkerThr ead.runIt(LeaderFollowerWorkerThread.java:80)
at org.apache.tomcat.util.threads.ThreadPool$ControlR unnable.run(ThreadPool.java:684)
at java.lang.Thread.run(Thread.java:595)
Caused by: javax.naming.NamingException: [LDAP: error code 80 - index generation failed]; remaining name 'uid=shouldnotseeme,ou=people,dc=clean-wire,dc=net'
at com.sun.jndi.ldap.LdapCtx.mapErrorCode(LdapCtx.jav a:3029)
at com.sun.jndi.ldap.LdapCtx.processReturnCode(LdapCt x.java:2931)
at com.sun.jndi.ldap.LdapCtx.processReturnCode(LdapCt x.java:2737)
at com.sun.jndi.ldap.LdapCtx.c_createSubcontext(LdapC tx.java:770)
at com.sun.jndi.toolkit.ctx.ComponentDirContext.p_cre ateSubcontext(ComponentDirContext.java:319)
at com.sun.jndi.toolkit.ctx.PartialCompositeDirContex t.createSubcontext(PartialCompositeDirContext.java :248)
at com.sun.jndi.toolkit.ctx.PartialCompositeDirContex t.createSubcontext(PartialCompositeDirContext.java :236)
at javax.naming.directory.InitialDirContext.createSub context(InitialDirContext.java:178)
at com.zimbra.cs.account.ldap.LdapUtil.simpleCreate(L dapUtil.java:433)
at com.zimbra.cs.account.ldap.LdapProvisioning.addAli asInternal(LdapProvisioning.java:1005)
... 24 more

Error code: service.FAILURE
Method: ZmCsfeCommand.prototype.invoke
Details:soap:Receiver

[jholder]

It looks like a permissions issue to me. Try
su root
chown -R zimbra:zimbra /opt/zimbra/*
cd /opt/zimbra/libexec
./zmfixperms

Then try to reinstall again.

[brained]

Originally Posted by wannabetenor

It looks like a permissions issue to me. Try
su root
chown -R zimbra:zimbra /opt/zimbra/*
cd /opt/zimbra/libexec
./zmfixperms

Then try to reinstall again.

Thanks wannbetenor, that unto itself was not the full resolution but I did use that as part of it.

A few hours of googling revealed that the BerkDB was probably corrupted.
http://www.openldap.org/lists/openld.../msg00243.html
http://svn.haxx.se/users/archive-2006-03/1026.shtml

Must have not had a clean shutdown just before I ran the backup off of the old server. Rsyncing that backup over to the new server carried the corruption with it.

So I went to the last backup just before the old server was shutdown, copied all of the openldap folders out of /opt/zimbra/ and dropped them onto the new server. Then I ran wannbetnors advice (fixperms and reinstall/upgrade) and everything is happiness now.

LVM, rsnapshot, rsync, google, and Zimbra forums are your best friends!