Build from sources on FreeBSD

[biloute]

Hi Zimbra people,

I'm trying to build Zimbra from sources on FreeBSD 6.1. Linux binary compatibility and rpms are not really an option for me (and there is no reason for zimbra not beeing able to be built/installed on this os ).
I followed intructions given in this post, and everything goes fine, up to the ant reset-the-world-stage1 step.
Here is what I get :

# ant reset-the-world-stage1
Buildfile: build.xml

clean:

clean:
   [delete] Deleting directory /usr/home/julien/installs/zimbra-install/ZimbraCommon/build

clean:
   [delete] Deleting directory /usr/home/julien/installs/zimbra-install/ZimbraIM/build
   [delete] Deleting directory /usr/home/julien/installs/zimbra-install/ZimbraServer/build

build-init:
    [mkdir] Created dir: /usr/home/julien/installs/zimbra-install/ZimbraServer/build/classes
    [mkdir] Created dir: /usr/home/julien/installs/zimbra-install/ZimbraServer/build/generated/src/java
    [mkdir] Created dir: /usr/home/julien/installs/zimbra-install/ZimbraServer/build/wiki
    [mkdir] Created dir: /usr/home/julien/installs/zimbra-install/ZimbraServer/build/data
Trying to override old definition of task deploy

make-dirs:
     [echo] Creating dirs under /usr/home/julien/installs/zimbra-install/ZimbraServer/build/dist
    [mkdir] Created dir: /usr/home/julien/installs/zimbra-install/ZimbraServer/build/dist/conf
    [mkdir] Created dir: /usr/home/julien/installs/zimbra-install/ZimbraServer/build/dist/openldap/etc/openldap/schema
    [mkdir] Created dir: /usr/home/julien/installs/zimbra-install/ZimbraServer/build/dist/bin
    [mkdir] Created dir: /usr/home/julien/installs/zimbra-install/ZimbraServer/build/dist/conf/msgs
    [mkdir] Created dir: /usr/home/julien/installs/zimbra-install/ZimbraServer/build/dist/db
    [mkdir] Created dir: /usr/home/julien/installs/zimbra-install/ZimbraServer/build/dist/lib/ext
    [mkdir] Created dir: /usr/home/julien/installs/zimbra-install/ZimbraServer/build/dist/lib/jars
    [mkdir] Created dir: /usr/home/julien/installs/zimbra-install/ZimbraServer/build/dist/libexec
    [mkdir] Created dir: /usr/home/julien/installs/zimbra-install/ZimbraServer/build/dist/tomcat/common/endorsed
    [mkdir] Created dir: /usr/home/julien/installs/zimbra-install/ZimbraServer/build/dist/tomcat/common/lib
    [mkdir] Created dir: /usr/home/julien/installs/zimbra-install/ZimbraServer/build/dist/tomcat/conf
    [mkdir] Created dir: /usr/home/julien/installs/zimbra-install/ZimbraServer/build/dist/tomcat/webapps
    [mkdir] Created dir: /usr/home/julien/installs/zimbra-install/ZimbraServer/build/dist/wiki

compile:

build-init:
    [mkdir] Created dir: /usr/home/julien/installs/zimbra-install/ZimbraCommon/build/classes

compile:
    [javac] Compiling 71 source files to /usr/home/julien/installs/zimbra-install/ZimbraCommon/build/classes
    [javac] Note: Some input files use or override a deprecated API.
    [javac] Note: Recompile with -Xlint:deprecation for details.
    [javac] Note: Some input files use unchecked or unsafe operations.
    [javac] Note: Recompile with -Xlint:unchecked for details.

jar:
      [jar] Building jar: /usr/home/julien/installs/zimbra-install/ZimbraCommon/build/zimbracommon.jar

build-init:
    [mkdir] Created dir: /usr/home/julien/installs/zimbra-install/ZimbraIM/build/classes

compile:
    [javac] Compiling 371 source files to /usr/home/julien/installs/zimbra-install/ZimbraIM/build/classes
    [javac] Note: Some input files use or override a deprecated API.
    [javac] Note: Recompile with -Xlint:deprecation for details.
    [javac] Note: Some input files use unchecked or unsafe operations.
    [javac] Note: Recompile with -Xlint:unchecked for details.

jar:
      [jar] Building jar: /usr/home/julien/installs/zimbra-install/ZimbraIM/build/zimbraim.jar
    [javac] Compiling 1052 source files to /usr/home/julien/installs/zimbra-install/ZimbraServer/build/classes
    [javac] Note: Some input files use or override a deprecated API.
    [javac] Note: Recompile with -Xlint:deprecation for details.
    [javac] Note: Some input files use unchecked or unsafe operations.
    [javac] Note: Recompile with -Xlint:unchecked for details.

set-buildinfo-vars:
     [echo] Version: 4.5

generate-buildinfo:
    [mkdir] Created dir: /usr/home/julien/installs/zimbra-install/ZimbraServer/build/buildinfo
    [javac] Compiling 1 source file to /usr/home/julien/installs/zimbra-install/ZimbraServer/build/classes

jar:
     [echo] Hostname: iono.niv2.com
      [jar] Building jar: /usr/home/julien/installs/zimbra-install/ZimbraServer/build/zimbrastore.jar

set-buildinfo-vars:
     [echo] Version: 4.5

generate-ldap-config:
    [mkdir] Created dir: /usr/home/julien/installs/zimbra-install/ZimbraServer/build/attrs
     [copy] Copying 2 files to /usr/home/julien/installs/zimbra-install/ZimbraServer/build/attrs
    [mkdir] Created dir: /usr/home/julien/installs/zimbra-install/ZimbraServer/build/ldap-config
     [java] [] WARN: local config file `/opt/zimbra/conf/localconfig.xml' is not readable
     [java] [] WARN: local config file `/opt/zimbra/conf/localconfig.xml' is not readable
     [java] [] WARN: local config file `/opt/zimbra/conf/localconfig.xml' is not readable
     [copy] Copying 1 file to /usr/home/julien/installs/zimbra-install/ZimbraServer/build/ldap-config
     [copy] Copying 1 file to /usr/home/julien/installs/zimbra-install/ZimbraServer/build/ldap-config
     [copy] Copying 1 file to /usr/home/julien/installs/zimbra-install/ZimbraServer/build/ldap-config

stop-webserver:

stop-tomcat-windows:

stop-tomcat-unix:

BUILD FAILED
/usr/home/julien/installs/zimbra-install/ZimbraServer/build.xml:1030: The following error occurred while executing this line:
/usr/home/julien/installs/zimbra-install/ZimbraServer/build.xml:938: Execute failed: java.io.IOException: java.io.IOException: /opt/zimbra/bin/tomcat: not found

Total time: 31 seconds

I'm not an expert at ant stuff, but as far as I can understand, it acts like make. It looks like something is messed up, as ant is looking for /opt/zimbra/bin/tomcat and /opt/zimbra/conf/localconfig.xml, but /opt/zimbra/bin and /opt/zimbra/conf does not even exist at this time, and even the ZimbraServer/build/dist/bin/ and conf/ directories are empty at this stage...

I'm using diablo-jdk1.5.0 (official port of Sun's jdk), and 3rd party software are build using --prefix=/opt/zimbra/.

Do you have any clues ?

Thanx

Julien

[KevinH]

You need to have tomcat installed in /opt/zimbra/tomcat and a start/stop script called /opt/zimbra/bin/tomcat for reset-the-world to complete

[biloute]

Thanx Kevin. I got it working by removing the stop-tomcat dependency of the reset-the-world-stage1 target for the firt run, to get the /opt/zimbra/bin/ scripts installed.
But now I'm facing a bit more weird issue with tomcat, wich I'm supposed to startup between the 2 stages.
Tomcat complains about not beeing able to find zimbra-native, despite the fact that -Djava.library.path=/opt/zimbra/lib/jars/ is defined in the tomcat startup script (and zimbra*.jar are present in this directory).
Here is what I get when I start tomcat manually (to make sure my problem is not script related, but I get the same result with /opt/zimbra/bin/tomcat) :

/usr/local/diablo-jdk1.5.0/bin/java 
-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager 
-Djava.util.logging.config.file=/var/opt/zimbra/apache-tomcat-5.5.20/conf/logging.properties 
-Djava.library.path=/var/opt/zimbra/lib/jars/ 
-Djava.endorsed.dirs=/var/opt/zimbra/apache-tomcat-5.5.20/common/endorsed -classpath :/var/opt/zimbra/apache-tomcat-5.5.20/bin/bootstrap.jar:/var/opt/zimbra/apache-tomcat-5.5.20/bin/commons-logging-api.jar 
-Dcatalina.base=/var/opt/zimbra/apache-tomcat-5.5.20 
-Dcatalina.home=/var/opt/zimbra/apache-tomcat-5.5.20 
-Djava.io.tmpdir=/var/opt/zimbra/apache-tomcat-5.5.20/temp 
org.apache.catalina.startup.Bootstrap start

Feb 24, 2007 11:59:47 AM org.apache.coyote.http11.Http11BaseProtocol init
INFO: Initializing Coyote HTTP/1.1 on http-7070
Feb 24, 2007 11:59:47 AM org.apache.coyote.http11.Http11BaseProtocol init
INFO: Initializing Coyote HTTP/1.1 on http-7443
Feb 24, 2007 11:59:47 AM org.apache.coyote.http11.Http11BaseProtocol init
INFO: Initializing Coyote HTTP/1.1 on http-7071
Feb 24, 2007 11:59:47 AM org.apache.catalina.startup.Catalina load
INFO: Initialization processed in 1397 ms
Feb 24, 2007 11:59:47 AM org.apache.catalina.core.StandardService start
INFO: Starting service Catalina
Feb 24, 2007 11:59:47 AM org.apache.catalina.core.StandardEngine start
INFO: Starting Servlet Engine: Apache Tomcat/5.5.20
Feb 24, 2007 11:59:48 AM org.apache.catalina.core.StandardHost start
INFO: XML validation disabled
Feb 24, 2007 11:59:48 AM org.apache.catalina.startup.HostConfig deployWAR
INFO: Deploying web application archive service.war
log4j:WARN No appenders could be found for logger (org.apache.commons.digester.Digester.sax).
log4j:WARN Please initialize the log4j system properly.
Zimbra server reserving server socket port=7110 bindaddr=null ssl=false
Zimbra server reserving server socket port=7995 bindaddr=null ssl=true
Zimbra server reserving server socket port=7143 bindaddr=null ssl=false
Zimbra server reserving server socket port=7993 bindaddr=null ssl=true
Fatal error: terminating: Failed to loadLibrary(zimbra-native)
java.lang.UnsatisfiedLinkError: no zimbra-native in java.library.path
        at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1682)
        at java.lang.Runtime.loadLibrary0(Runtime.java:822)
        at java.lang.System.loadLibrary(System.java:992)
        at com.zimbra.znative.Util.loadLibrary(Util.java:54)
        at com.zimbra.znative.Util.<clinit>(Util.java:71)
        at com.zimbra.znative.Process.geteuid(Process.java:41)
        at com.zimbra.cs.servlet.PrivilegedServlet.init(PrivilegedServlet.java:138)
        at javax.servlet.GenericServlet.init(GenericServlet.java:211)
        at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1105)
        at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:932)
        at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:3951)
        at org.apache.catalina.core.StandardContext.start(StandardContext.java:4225)
        at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:759)
        at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:739)
        at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:524)
        at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:809)
        at org.apache.catalina.startup.HostConfig.deployWARs(HostConfig.java:698)
        at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:472)
        at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1122)
        at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:310)
        at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:119)
        at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1021)
        at org.apache.catalina.core.StandardHost.start(StandardHost.java:718)
        at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1013)
        at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:442)
        at org.apache.catalina.core.StandardService.start(StandardService.java:450)
        at org.apache.catalina.core.StandardServer.start(StandardServer.java:709)
        at org.apache.catalina.startup.Catalina.start(Catalina.java:551)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:585)
        at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:294)
        at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:432)

(pls note that on my system, /opt is a symlink to /var/opt, and /opt/zimbra/tomcat symlinks to /opt/zimbra/apache-tomcat-5.5.20)

I'm a bit puzzled here... do you have any idea ?

Thanx

Julien

[SuperMario]

While I'm not 100% sure about this solution, this could be an issue of putting things in the PATH variable versus putting them in the CLASSPATH variable. I've run into this problem before with ant, but I can't remember how I fixed it. I wanted to go ahead and post what I thought it was, and I'll try to post again if I remember what I think I did to fix this.

[biloute]

Hi monvural,

Thanx for you hint, it bring me in the right direction Just in case it would be useful to someone, I had to add /opt/zimbra/lib/jars/zimbracommon.jar to the classpath in my tomcat startup script to make it working.
The weird thing is that the zimbra tomcat startup script and tomcat's error message let you suppose it has to be present in java.library.path, not in classpath. Maybe some platform specific issue here ?

Julien

[biloute]

One more problem with tomcat starting up... this time I get strange IllegalAccessError :

Feb 27, 2007 3:52:06 PM org.apache.coyote.http11.Http11BaseProtocol init
INFO: Initializing Coyote HTTP/1.1 on http-7070
Feb 27, 2007 3:52:06 PM org.apache.coyote.http11.Http11BaseProtocol init
INFO: Initializing Coyote HTTP/1.1 on http-7443
Feb 27, 2007 3:52:06 PM org.apache.coyote.http11.Http11BaseProtocol init
INFO: Initializing Coyote HTTP/1.1 on http-7071
Feb 27, 2007 3:52:06 PM org.apache.catalina.startup.Catalina load
INFO: Initialization processed in 1384 ms
Feb 27, 2007 3:52:06 PM org.apache.catalina.core.StandardService start
INFO: Starting service Catalina
Feb 27, 2007 3:52:06 PM org.apache.catalina.core.StandardEngine start
INFO: Starting Servlet Engine: Apache Tomcat/5.5.20
Feb 27, 2007 3:52:07 PM org.apache.catalina.core.StandardHost start
INFO: XML validation disabled
Feb 27, 2007 3:52:07 PM org.apache.catalina.startup.HostConfig deployWAR
INFO: Deploying web application archive service.war
Feb 27, 2007 3:52:08 PM org.apache.catalina.startup.HostConfig deployWAR
SEVERE: Error deploying web application archive service.war
org.apache.commons.logging.LogConfigurationException: java.lang.IllegalAccessError: tried to access class org.apache.log4j.RollingCalendar from class org.apache.log4j.ZimbraStatsAppender (Caused by java.lang.IllegalAccessError: tried to access class org.apache.log4j.RollingCalendar from class org.apache.log4j.ZimbraStatsAppender)
        at org.apache.commons.logging.impl.LogFactoryImpl.newInstance(LogFactoryImpl.java:538)
        at org.apache.commons.logging.impl.LogFactoryImpl.getInstance(LogFactoryImpl.java:235)
        at org.apache.commons.logging.LogFactory.getLog(LogFactory.java:370)
        at org.apache.catalina.core.ContainerBase.getLogger(ContainerBase.java:380)
        at org.apache.catalina.core.StandardContext.start(StandardContext.java:4114)
        at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:759)
        at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:739)
        at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:524)
        at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:809)
        at org.apache.catalina.startup.HostConfig.deployWARs(HostConfig.java:698)
        at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:472)
        at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1122)
        at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:310)
        at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:119)
        at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1021)
        at org.apache.catalina.core.StandardHost.start(StandardHost.java:718)
        at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1013)
        at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:442)
        at org.apache.catalina.core.StandardService.start(StandardService.java:450)
        at org.apache.catalina.core.StandardServer.start(StandardServer.java:709)
        at org.apache.catalina.startup.Catalina.start(Catalina.java:551)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:585)
        at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:294)
        at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:432)
Caused by: java.lang.IllegalAccessError: tried to access class org.apache.log4j.RollingCalendar from class org.apache.log4j.ZimbraStatsAppender
        at org.apache.log4j.ZimbraStatsAppender.<init>(ZimbraStatsAppender.java:81)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:494)
        at java.lang.Class.newInstance0(Class.java:350)
        at java.lang.Class.newInstance(Class.java:303)
        at org.apache.log4j.helpers.OptionConverter.instantiateByClassName(OptionConverter.java:319)
        at org.apache.log4j.helpers.OptionConverter.instantiateByKey(OptionConverter.java:112)
        at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:620)
        at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:603)
        at org.apache.log4j.PropertyConfigurator.parseCatsAndRenderers(PropertyConfigurator.java:524)
        at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:408)
        at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:432)
        at org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:460)
        at org.apache.log4j.LogManager.<clinit>(LogManager.java:113)
        at org.apache.log4j.Logger.getLogger(Logger.java:85)
        at org.apache.commons.logging.impl.Log4JLogger.<init>(Log4JLogger.java:102)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:494)
        at org.apache.commons.logging.impl.LogFactoryImpl.newInstance(LogFactoryImpl.java:529)
        ... 26 more
Feb 27, 2007 3:52:08 PM org.apache.coyote.http11.Http11BaseProtocol start
INFO: Starting Coyote HTTP/1.1 on http-7070
Feb 27, 2007 3:52:08 PM org.apache.coyote.http11.Http11BaseProtocol start
INFO: Starting Coyote HTTP/1.1 on http-7443
Feb 27, 2007 3:52:08 PM org.apache.coyote.http11.Http11BaseProtocol start
INFO: Starting Coyote HTTP/1.1 on http-7071
Feb 27, 2007 3:52:08 PM org.apache.catalina.startup.Catalina start
INFO: Server startup in 1913 ms

The more I try solving problem, the more I get even more serious ones

If you have any idea...

Thanx

[vshah]

Are you starting from the patched version the dwhite had posted a link to in the forums?

[biloute]

Originally Posted by vshah

Are you starting from the patched version the dwhite had posted a link to in the forums?

Wow, I'm wondering how I missed this patchset
I'll give it a try tomorow, thanx for your pointer.
To answer you : no, I'm building from fresh subversion checkout and latest version of 3rd party software.

Thanx,
Julien