fc4 zcs-3.0.1 -> 3.1 upgrade worked great.. almost

[keyhman]

Hi. I read some posts, didn't see anything scary so I ran the upgrade on my mail server -> it took a long time to upgrade everything but it seemed to work without any errors, just long pauses as it imported my saved configuration.

But..the key feature I was after in the upgrade - mail queue management doesn't work. When I click on "Mail Queues" : get an error msg pop-up with this message:

Message: system failure: exception during auth {RemoteManager: mx1.example.com->zimbra@mx1.example.com:22}
com.zimbra.cs.service.ServiceException: system failure: exception during auth {RemoteManager: mx1.example.com->zimbra@mx1.example.com:22}
at com.zimbra.cs.service.ServiceException.FAILURE(Ser viceException.java:131)
at com.zimbra.cs.rmgmt.RemoteManager.getSession(Remot eManager.java:192)
at com.zimbra.cs.rmgmt.RemoteManager.execute(RemoteMa nager.java:129)
at com.zimbra.cs.service.admin.GetMailQueueInfo.handl e(GetMailQueueInfo.java:55)
at com.zimbra.soap.SoapEngine.dispatchRequest(SoapEng ine.java:255)
at com.zimbra.soap.SoapEngine.dispatch(SoapEngine.jav a:163)
at com.zimbra.soap.SoapEngine.dispatch(SoapEngine.jav a:84)
at com.zimbra.soap.SoapServlet.doPost(SoapServlet.jav a:228)
at javax.servlet.http.HttpServlet.service(HttpServlet .java:709)
at com.zimbra.cs.servlet.ZimbraServlet.service(Zimbra Servlet.java:154)
at javax.servlet.http.HttpServlet.service(HttpServlet .java:802)
at org.apache.catalina.core.ApplicationFilterChain.in ternalDoFilter(ApplicationFilterChain.java:252)
at org.apache.catalina.core.ApplicationFilterChain.do Filter(ApplicationFilterChain.java:173)
at org.apache.catalina.core.StandardWrapperValve.invo ke(StandardWrapperValve.java:213)
at org.apache.catalina.core.StandardContextValve.invo ke(StandardContextValve.java:178)
at org.apache.catalina.core.StandardHostValve.invoke( StandardHostValve.java:126)
at org.apache.catalina.valves.ErrorReportValve.invoke (ErrorReportValve.java:105)
at org.apache.catalina.core.StandardEngineValve.invok e(StandardEngineValve.java:107)
at org.apache.catalina.valves.AccessLogValve.invoke(A ccessLogValve.java:541)
at org.apache.catalina.connector.CoyoteAdapter.servic e(CoyoteAdapter.java:148)
at org.apache.coyote.http11.Http11Processor.process(H ttp11Processor.java:869)
at org.apache.coyote.http11.Http11BaseProtocol$Http11 ConnectionHandler.processConnection(Http11BaseProt ocol.java:667)
at org.apache.tomcat.util.net.PoolTcpEndpoint.process Socket(PoolTcpEndpoint.java:527)
at org.apache.tomcat.util.net.LeaderFollowerWorkerThr ead.runIt(LeaderFollowerWorkerThread.java:80)
at org.apache.tomcat.util.threads.ThreadPool$ControlR unnable.run(ThreadPool.java:684)
at java.lang.Thread.run(Thread.java:595)
Caused by: java.io.IOException: auth failed
at com.zimbra.cs.rmgmt.RemoteManager.getSession(Remot eManager.java:185)
... 24 more

Error code: service.FAILURE
Method: ZmCsfeCommand.prototype.invoke
Details:soap:Receiver


Sorry for the long msg but I haven't seen anyone else refernce a problem like this since the upgrade was released. any ideas?

steve.

[keyhman]

If I'm upgrading from previous version is there a new remote management command that wasn't a feature in previous versions?

From what I can see zmrcd == sshd and zmrc == ssh -c

If that's true, then there's a conflict. I use ssh for remote systems management on port 22. I've changed other ports on the original install to avoid conflicts with apache, etc...

Do I need to use zmprov to change the remote management port to an unused value to fix the mail queues error message I still get from admin gui?

steve

[keyhman]

On a hunch I tried using this:
zmprov ms mx1.example.com zimbraRemoteManagementPort 8022

zmcontrol stop && zmcontrol start

I tried the mail queues gui again and The error message changed to show the port number modification I made:

Message: system failure: exception during auth {RemoteManager: mx1.example->zimbra@mx1.example.com:8022}
com.zimbra.cs.service.ServiceException: system failure: exception during auth {RemoteManager: mx1.example.com->zimbra@mx1.example.com:8022}
at com.zimbra.cs.service.ServiceException.FAILURE(Ser viceException.java:131)
at

.....

Caused by: java.io.IOException: There was a problem while talking to mx1.example:8022
at ch.ethz.ssh2.Connection.connect(Connection.java:64 2)
at ch.ethz.ssh2.Connection.connect(Connection.java:46 0)
at com.zimbra.cs.rmgmt.RemoteManager.getSession(Remot eManager.java:183)
... 24 more
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl .java:333)
at java.net.PlainSocketImpl.connectToAddress(PlainSoc ketImpl.java:195)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.j ava:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.j ava:366)
at java.net.Socket.connect(Socket.java:507)
at ch.ethz.ssh2.transport.TransportManager.initialize (TransportManager.java:299)
at ch.ethz.ssh2.Connection.connect(Connection.java:59 1)
... 26 more

Error code: service.FAILURE
Method: ZmCsfeCommand.prototype.invoke
Details:soap:Receiver

This is obviously different from the first error -- is it possible that the remote management relies on the system ssh daemon ?

please clarify.

steve.

[keyhman]

Working on the assumption that zimbra relies on the system ssh daemon (Based on previous posted msgs) I've reverted back to port 22 in the server configs for my host:

zmprov ms mx1.example.com zimbraRemoteManagementPort 22

zmcontrol stop && zmcontrol start

and the message reverted back to the original error -- hence my conclusion that the system ssh daemon is part of this puzzle.

it seems to me like the ssh daemon isn't recognizing or accepting the certificate found in ~zimbra/.ssh/zimbra_identity

can anyone clarify and advise on a fix?

steve.

[keyhman]

I found the solution.

If you read the man page to ssh and sshd , you'll find that for PubKey auth, the host must be authorized for this method of auth. That requires the host public key to be present in ~zimbra/.ssh/authorized_hosts and ~zimbra/.ssh/known_hosts

I manually copied the contents of ~zimbra/.ssh/zimbra_identity.pub to these two files and the queue manager started working immediately. no more errors.

Great new feature guys. keep up the good work.

steve

[KevinH]

Thanks. Can you file this in bugzilla?

[adobrin]

i don't seem to have an executable 'zmqstat' .. where should this file be?

Message: system failure: exception executing command: zmqstat all with {RemoteManager: mail.domain.com->zimbra@mail.domain.com:22}
com.zimbra.cs.service.ServiceException: system failure: exception executing command: zmqstat all with {RemoteManager: mail.domain.com->zimbra@mail.domain.com:22}
at com.zimbra.cs.service.ServiceException.FAILURE(Ser viceException.java:131)
at com.zimbra.cs.rmgmt.RemoteManager.execute(RemoteMa nager.java:161)
at com.zimbra.cs.service.admin.GetMailQueueInfo.handl e(GetMailQueueInfo.java:55)
at com.zimbra.soap.SoapEngine.dispatchRequest(SoapEng ine.java:255)
at com.zimbra.soap.SoapEngine.dispatch(SoapEngine.jav a:163)
at com.zimbra.soap.SoapEngine.dispatch(SoapEngine.jav a:84)
at com.zimbra.soap.SoapServlet.doPost(SoapServlet.jav a:228)
at javax.servlet.http.HttpServlet.service(HttpServlet .java:709)
at com.zimbra.cs.servlet.ZimbraServlet.service(Zimbra Servlet.java:154)
at javax.servlet.http.HttpServlet.service(HttpServlet .java:802)
at org.apache.catalina.core.ApplicationFilterChain.in ternalDoFilter(ApplicationFilterChain.java:252)
at org.apache.catalina.core.ApplicationFilterChain.do Filter(ApplicationFilterChain.java:173)
at org.apache.catalina.core.StandardWrapperValve.invo ke(StandardWrapperValve.java:213)
at org.apache.catalina.core.StandardContextValve.invo ke(StandardContextValve.java:178)
at org.apache.catalina.core.StandardHostValve.invoke( StandardHostValve.java:126)
at org.apache.catalina.valves.ErrorReportValve.invoke (ErrorReportValve.java:105)
at org.apache.catalina.core.StandardEngineValve.invok e(StandardEngineValve.java:107)
at org.apache.catalina.valves.AccessLogValve.invoke(A ccessLogValve.java:541)
at org.apache.catalina.connector.CoyoteAdapter.servic e(CoyoteAdapter.java:148)
at org.apache.coyote.http11.Http11Processor.process(H ttp11Processor.java:869)
at org.apache.coyote.http11.Http11BaseProtocol$Http11 ConnectionHandler.processConnection(Http11BaseProt ocol.java:667)
at org.apache.tomcat.util.net.PoolTcpEndpoint.process Socket(PoolTcpEndpoint.java:527)
at org.apache.tomcat.util.net.LeaderFollowerWorkerThr ead.runIt(LeaderFollowerWorkerThread.java:80)
at org.apache.tomcat.util.threads.ThreadPool$ControlR unnable.run(ThreadPool.java:684)
at java.lang.Thread.run(Thread.java:595)
Caused by: java.io.IOException: command failed127
at com.zimbra.cs.rmgmt.RemoteManager.execute(RemoteMa nager.java:156)
... 23 more

Error code: service.FAILURE
Method: ZmCsfeCommand.prototype.invoke
Details:soap:Receiver

[KevinH]

Originally Posted by adobrin

i don't seem to have an executable 'zmqstat' .. where should this file be?

libexec/zmqstat

[adobrin]

[zimbra@mail libexec]$ ./zmqstat
Can't chdir to /opt/zimbra/postfix-2.2.9/spool/incoming at ./zmqstat line 182.
[zimbra@mail libexec]$

[adobrin]

which user is it supposed to be logging in as?

spool/* are all postfix owned, and most of them chmod 400 (ish).