ZCS 6.0.2 Admin Interface Missing Features and/or Broken

[Guenther983]

I just ran an installation upgrade from 5.0.9 to 6.0.2 on Ubuntu 6 - I figured waiting to the 3rd release of v6 would flush out some obvious bugs - I was wrong.

Fortunately it appears that my users are able to send and receive email and the Outlook and mobile connectors are working as they should.

I am however running into a wide variety of issues in the admin UI that indicates the installation was less than smooth.

Immediately I noticed some missing things from the UI...
* The Global Settings License tab is completely missing
* The Backup/Restore tool option is completely missing
* The Features options do not list the Mobile option
* The Download files doesn't show the Outlook Connector

It appears that much of this falls under the zimlets category. But they appear to be installed...

~/% zmzimletctl listZimlets all
Installed Zimlet files on this host:
Installed Zimlets in LDAP:
com_zimbra_backuprestore (ext)
com_zimbra_cert_manager (ext)
com_zimbra_cluster (ext)
com_zimbra_convertd (ext)
com_zimbra_date
com_zimbra_domainadmin (ext)
com_zimbra_email
com_zimbra_hsm (ext)
com_zimbra_license (ext)
com_zimbra_local
com_zimbra_mobilesync (ext)
com_zimbra_phone
com_zimbra_url
com_zimbra_ymemoticons
Available Zimlets in COS:
default:
com_zimbra_date
com_zimbra_email
com_zimbra_local
com_zimbra_phone
com_zimbra_url
com_zimbra_ymemoticons

I have verified via the command line that the zimbra license is installed, but I have real concern that if I go to edit or add a new user that user will lose the ability to use the Outlook connector or Mobile access because the UI isn't showing that as an option.


In addition to the missing features..

The Server Status is showing that all services are down.

The Server Stats are returning errors. This was likely because the logger service was FAILING to start up.

Digging through a variety of log files and this wiki article... http://wiki.zimbra.com/index.php?tit...x_and_later%29

... I determined that the problem was likely a missing sqlite database in the /opt/zimbra/logger/db/data - in fact there was no "data" folder in the "db" directory. When I manually created that folder - a sqlite file is created in that folder but it is missing any data.

I am hoping for help on any and all of these issues because I am stuck in a no-mans land on this upgrade. Email is working properly but I am afraid that it may stop working.

[phoenix]

Can you confirm that you've installed the correct NE version during the upgrade? What file did you download? What is the output of the following commands:

su - zimbra
zmcontrol -v
zmlicense -p

[Guenther983]

Yes I can confirm that we downloaded the Network Edition. And as the previous post said - the Outlook Connector is working as well as the Mobile Sync option and the zimlets claim to be installed. They just are not visible in the UI along with the other logging/stats issues.

zcs-NETWORK-6.0.2_GA_1912.UBUNTU6.20091020163111.tgz

zimbra@zimbra:~$ zmcontrol -v

Release 6.0.2_GA_1912.UBUNTU6 UBUNTU6 NETWORK edition.

zimbra@zimbra:~$ zmlicense -p

AccountsLimit=100
ValidFrom=20081012000000Z
HierarchicalStorageManagementEnabled=true
MobileSyncAccountsLimit=100
BackupEnabled=true
CrossMailboxSearchEnabled=true
ISyncAccountsLimit=100
InstallType=regular
IssuedToEmail=[omitted from forum]
MobileSyncEnabled=true
ValidUntil=20091231170000Z
IssuedToName=[omitted from forum]
LicenseId=[omitted from forum]
ResellerName=
MAPIConnectorAccountsLimit=100
AttachmentConversionEnabled=true
IssuedOn=20081013170234Z

[Guenther983]

Ok we have figured out the Zimlet issue... it appears that it originates from a top level security issue on the /opt/zimbra folder. On our server it is configured as...

drwxr-xr-x 58 root root 4096 Oct 26 12:10 zimbra

So it is owned by root with no write access to the zimbra user. NOTE: We had run zmfixperms many times and it never fixed this.

We attempted to force redeploy of the license zimlet and ran...
% zmzimletctl deploy zimlets-network/com_zimbra_license.zip

The log file reported the following java stack trace...

2009-10-26 11:35:44,152 INFO [Thread-22365] [] zimlet - deploy
java.io.IOException: No such file or directory
at java.io.UnixFileSystem.createFileExclusively(Nativ e Method)
at java.io.File.createNewFile(File.java:883)
at com.zimbra.cs.zimlet.ZimletUtil.writeFile(ZimletUt il.java:610)
at com.zimbra.cs.zimlet.ZimletUtil.installZimlet(Ziml etUtil.java:557)
at com.zimbra.cs.zimlet.ZimletUtil.deployZimlet(Zimle tUtil.java:486)
at com.zimbra.cs.zimlet.ZimletUtil.deployZimlet(Zimle tUtil.java:420)
at com.zimbra.cs.service.admin.DeployZimlet$DeployThr ead.run(DeployZimlet.java:98)
at java.lang.Thread.run(Thread.java:619)

After poking around finding the right jars and decompiling we determined that the code in question was trying to unzip the com_zimbra_license.zip file into various folders one of which was /opt/zimbra/zimlets-properties (which it would attempt to create if it didn't exist). But because the zimbra user had no write access to this folder it failed.

After manually creating this folder and setting permissions on it - we were able to force redeploy all of the relevant zimlets resulting in this...

zimbra@zimbra:~$ zmzimletctl listZimlets all
Installed Zimlet files on this host:
com_zimbra_adminversioncheck (ext)
com_zimbra_backuprestore (ext)
com_zimbra_bulkprovision (ext)
com_zimbra_cert_manager (ext)
com_zimbra_convertd (ext)
com_zimbra_date
com_zimbra_delegatedadmin (ext)
com_zimbra_email
com_zimbra_hsm (ext)
com_zimbra_license (ext)
com_zimbra_local
com_zimbra_mobilesync (ext)
com_zimbra_phone
com_zimbra_url
com_zimbra_ymemoticons
Installed Zimlets in LDAP:
com_zimbra_adminversioncheck (ext)
com_zimbra_backuprestore (ext)
com_zimbra_bulkprovision (ext)
com_zimbra_cert_manager (ext)
com_zimbra_convertd (ext)
com_zimbra_date
com_zimbra_delegatedadmin (ext)
com_zimbra_email
com_zimbra_hsm (ext)
com_zimbra_license (ext)
com_zimbra_local
com_zimbra_mobilesync (ext)
com_zimbra_phone
com_zimbra_url
com_zimbra_ymemoticons
Available Zimlets in COS:
default:
com_zimbra_date
com_zimbra_email
com_zimbra_local
com_zimbra_phone
com_zimbra_url
com_zimbra_ymemoticons

Note that unlike the previous list - this shows the zimlets installed on both the host and LDAP server. Once we did a zmcontrol stop/start the zimlets all appeared.

So now we are left with the stats logging issue mentioned above, although I wonder if it is related to the security permissions on various folders. Specifically the /opt/zimbra folder still has suspect security and the subfolders on that are a mix and match of those owned by root and those owned by zimbra (see list below).

Does anyone know the correct configuration? (i.e. Should everything in this directory by owned by zimbra/zimbra or is it correct to have some owned by root. The obvious concern is that if we change the permissions globally we can break the server.

Ideally zmfixperms would be updated to check every folder and make sure it is correct.

zimbra@zimbra:~$ ls -l
total 240
lrwxrwxrwx 1 root root 29 Oct 24 12:01 amavisd -> /opt/zimbra/amavisd-new-2.6.4
drwxr-xr-x 4 root root 4096 Oct 24 12:01 amavisd-new-2.6.4
lrwxrwxrwx 1 root root 25 Oct 24 12:02 aspell -> /opt/zimbra/aspell-0.60.6
drwxr-xr-x 6 root root 4096 Oct 24 12:02 aspell-0.60.6
drwxr-xr-x 4 zimbra zimbra 4096 Oct 26 00:00 backup
lrwxrwxrwx 1 root root 24 Oct 24 12:01 bdb -> /opt/zimbra/bdb-4.7.25.4
drwxr-xr-x 6 root root 4096 Oct 24 12:01 bdb-4.7.25.4
drwxr-xr-x 2 root root 4096 Oct 24 12:01 bin
lrwxrwxrwx 1 root root 25 Oct 24 12:01 clamav -> /opt/zimbra/clamav-0.95.2
drwxr-xr-x 10 root root 4096 Oct 24 12:01 clamav-0.95.2
drwxr-xr-x 9 zimbra zimbra 4096 Oct 26 12:28 conf
-rw------- 1 root root 2721 Oct 24 12:12 config.1532
-rw------- 1 root root 2316 Sep 16 2008 config.18037
drwxr-xr-x 2 root root 4096 Oct 24 12:01 contrib
drwxr-xr-x 10 zimbra zimbra 4096 Oct 24 12:02 convertd
lrwxrwxrwx 1 root root 23 Oct 24 12:01 curl -> /opt/zimbra/curl-7.19.6
drwxr-xr-x 6 root root 4096 Oct 24 12:01 curl-7.19.6
lrwxrwxrwx 1 root root 32 Oct 24 12:01 cyrus-sasl -> /opt/zimbra/cyrus-sasl-2.1.23.3z
drwxr-xr-x 4 root root 4096 Oct 24 12:01 cyrus-sasl-2.1.22.3z
drwxr-xr-x 8 root root 4096 Oct 24 12:01 cyrus-sasl-2.1.23.3z
drwxr-xr-x 8 zimbra zimbra 4096 Oct 24 12:01 data
drwxr-xr-x 3 zimbra zimbra 4096 Oct 26 12:27 db
drwxr-xr-x 2 zimbra zimbra 4096 Oct 24 12:01 docs
lrwxrwxrwx 1 root root 23 Oct 24 12:01 dspam -> /opt/zimbra/dspam-3.8.0
drwxr-xr-x 7 root root 4096 Oct 24 12:01 dspam-3.8.0
drwxr-xr-x 2 zimbra zimbra 4096 Sep 16 2008 fbqueue
lrwxrwxrwx 1 root root 25 Oct 24 12:01 heimdal -> /opt/zimbra/heimdal-1.2.1
drwxr-xr-x 6 root root 4096 Oct 24 12:01 heimdal-1.2.1
lrwxrwxrwx 1 root root 24 Oct 24 12:02 httpd -> /opt/zimbra/httpd-2.2.11
drwxr-xr-x 15 root root 4096 Oct 24 12:02 httpd-2.2.11
drwxr-xr-x 3 zimbra zimbra 4096 Sep 16 2008 index
lrwxrwxrwx 1 root root 23 Oct 24 12:01 java -> /opt/zimbra/jdk1.6.0_16
drwxr-xr-x 3 root root 4096 Oct 24 12:01 jdk1.5.0_15
drwxr-xr-x 8 root root 4096 Oct 24 12:01 jdk1.6.0_16
lrwxrwxrwx 1 root root 27 Oct 24 12:02 jetty -> /opt/zimbra/jetty-6.1.15.z6
drwxr-xr-x 11 root root 4096 Oct 24 12:02 jetty-6.1.15.z6
drwxr-xr-x 5 root root 4096 Oct 24 12:01 jetty-6.1.5
lrwxrwxrwx 1 root root 28 Oct 24 12:02 keyview -> /opt/zimbra/keyview-10.8.1.0
drwxr-xr-x 6 root root 4096 Oct 24 12:02 keyview-10.8.1.0
drwxr-xr-x 6 root root 4096 Oct 24 12:01 lib
drwxr-xr-x 4 root root 4096 Oct 26 10:56 libexec
lrwxrwxrwx 1 root root 26 Oct 24 12:01 libtool -> /opt/zimbra/libtool-2.2.6a
drwxr-xr-x 4 root root 4096 Oct 24 12:01 libtool-2.2.6a
drwxr-xr-x 3 zimbra zimbra 12288 Oct 26 12:42 log
drwxr-xr-x 3 zimbra zimbra 4096 Oct 24 12:01 logger
lrwxrwxrwx 1 root root 27 Oct 24 12:02 mailboxd -> /opt/zimbra/jetty-6.1.15.z6
lrwxrwxrwx 1 root root 59 Oct 24 12:02 mysql -> /opt/zimbra/mysql-standard-5.0.85-pc-linux-gnu-i686-glibc23
drwxr-xr-x 7 root root 4096 Oct 24 12:01 mysql-standard-5.0.85-pc-linux-gnu-i686-glibc23
lrwxrwxrwx 1 root root 28 Oct 24 12:01 net-snmp -> /opt/zimbra/net-snmp-5.4.2.1
drwxr-xr-x 3 root root 4096 Oct 24 12:00 net-snmp-5.4.1
drwxr-xr-x 10 root root 4096 Oct 24 12:01 net-snmp-5.4.2.1
lrwxrwxrwx 1 root root 30 Oct 24 12:01 openldap -> /opt/zimbra/openldap-2.4.18.2z
drwxr-xr-x 9 root root 4096 Oct 24 12:01 openldap-2.4.18.2z
lrwxrwxrwx 1 root root 36 Oct 24 12:01 openldap-clibs -> /opt/zimbra/openldap-clibs-2.4.18.2z
drwxr-xr-x 3 root root 4096 Oct 24 12:01 openldap-clibs-2.4.18.2z
drwxr-xr-x 4 root root 4096 Dec 6 2008 openldap-data
lrwxrwxrwx 1 root root 26 Oct 24 12:01 openssl -> /opt/zimbra/openssl-0.9.8k
drwxr-xr-x 6 root root 4096 Oct 24 12:01 openssl-0.9.8k
lrwxrwxrwx 1 root root 28 Oct 24 12:01 postfix -> /opt/zimbra/postfix-2.6.5.2z
drwxr-xr-x 3 root root 4096 Oct 24 12:00 postfix-2.4.7.5z
drwxr-xr-x 6 root root 4096 Oct 24 12:01 postfix-2.6.5.2z
drwxr-xr-x 3 zimbra zimbra 4096 Oct 26 12:26 redolog
drwxr-xr-x 2 root root 4096 Jul 6 20:41 save-07012009
lrwxrwxrwx 1 root root 28 Oct 24 12:01 snmp -> /opt/zimbra/net-snmp-5.4.2.1
drwxr-xr-x 5 zimbra zimbra 4096 Sep 16 2008 ssl
drwxr-xr-x 4 zimbra zimbra 4096 Sep 16 2008 store
lrwxrwxrwx 1 root root 24 Oct 24 12:01 tcmalloc -> /opt/zimbra/tcmalloc-1.3
drwxr-xr-x 5 root root 4096 Oct 24 12:01 tcmalloc-1.3
drwxr-xr-x 3 zimbra zimbra 4096 Oct 24 12:02 wiki
drwxr-xr-x 6 root root 4096 Oct 24 12:01 zimbramon
drwxr-xr-x 2 zimbra zimbra 4096 Oct 24 12:02 zimlets
drwxr-xr-x 2 zimbra zimbra 4096 Oct 24 12:01 zimlets-admin-extra
drwxr-xr-x 17 zimbra zimbra 4096 Oct 26 12:26 zimlets-deployed
drwxr-xr-x 2 zimbra zimbra 4096 Oct 24 12:01 zimlets-experimental
drwxr-xr-x 2 zimbra zimbra 4096 Oct 24 12:01 zimlets-extra
drwxr-xr-x 2 zimbra zimbra 4096 Oct 24 12:01 zimlets-network
drwxr-xr-x 16 zimbra zimbra 4096 Oct 26 12:26 zimlets-properties
drwxr-xr-x 408 zimbra zimbra 12288 Oct 26 06:25 zmstat

[ArcaneMagus]

The permissions and owners in that output match the ones on my server so that should be good.

[Guenther983]

Thank you ArcaneMagus for verifying the folder perms.

I also confirmed with zimbra support that the base folder owned by root is not an issue and is actually correct. The problem with some of this is not all the required folders were properly created during the upgrade.

zmfixperms will fix the perms if that folder does exist - but it does so simply by looking for zimlets* folders - so if the folder isn't there it won't create it.

The logger/stats issue still persists for me. I discovered after a ton of trial and error I can delete the /opt/zimbra/logger/db/data/ folder - re-run zmfixperms (which will recreate the data subfolder) and then I can force "zmlogger" to run manually which will create some of the SQLite schema tables. But I am stuck at that point. zmlogger will not run on its own. If I manually run it and click on the UI I can see the UI talking to it but getting errors like "zimbra.[mydomain].com not found". So something is still fairly messed up here. Seems like the major rewrite of the stats logging wasn't particularly robustly tested.

[JeeZ]

I have the same issue with the unavailable License tab. Running 6.0.3, upgraded from 5.0.8, Network Edition. And in the download area of the admin pannel, the Outlook Connector disapeared. The old one is still working, but I would prefer running the current one.

Any clues?
Thanks!

[ericortego]

I also have what seems to be the same problem. At first most zimlets were missing, backuprestore, cert_manager, samba, posix, server statistics, and domainadmin.

undeploying them all and redeploying one at a time has many of them working again but I cannot find any way (even with zimbra support) to get server statistics, posixaccounts, or samba working.