[Script] ZCS 7.1.4 to ZCS 7.1.4 Migration script using any Distro. to any Distro.

[Eclipse]

Hey guys, I've been working slowly on a all in one solution for migrating from any distribution running ZCS 7.1.4 OSE to 7.1.4 OSE with any Architecture, meaning -

x32 CentOS to Ubuntu x64 vice Versa

Subsitute CentOS and Ubuntu with your choice, it will migrate all emails, all attachments, all calendars, all domains, all accounts, all passwords

I basically combined two guides into two separate bash scripts that do all of the work for you, these two guides are -

Platform and OS Independent ZCS to ZCS Migration Using Rsync - Zimbra :: Wiki
and -
Network Edition: Moving from 32-bit to 64-bit Server - Zimbra :: Wiki

Note, this script is not final and I do not advise anyone to use it on their production server at first glance, I want people to test it on their own with virtual machines or their own test servers and give feed back to let me know if something isn't right.

So what you need to do is copy both scripts I have posted below, one of them is Script1 and the other Script 2, I was shooting for an all in one script but I figured I might as well have it in two parts to avoid headaches.

This script ONLY does for the time being 1:1 migrations, meaning any server setups out there where you have an external LDAP server, proxy, etc, will not work.

This only migrates all in 1 Zimbra solutions.

To get your migration started, put Part 1 on the server you are migrating from and run it on there as Root, follow the prompts it asks you, it will then finish and ask you to run Script 2 on the new server or destination server you are migrating to.

Be advised, you must have a working vanilla installation with the exact same hostname, rsync installed, and Zimbra version installed on your new server BEFORE RUNNING SCRIPT 2

Now, I have not tested this, but if you migrate from 7.1.x to 7.1.4 it MAY work, I have not tested this, I'd like to know if this works.

So then put Script 2 on the destination and run as Root, be very sure to put the correct IP address in for the source server when it asks you. It'll then put the IP address you entered into a variable and use it for copies, you'll be prompted to enter the root password for each copy.

It uses Rsync to copy over your mail database, index, and store, it does not support ssh public/private keys, so you'll need direct SSH root access.

AFTER RUNNING SCRIPT 2 READ THIS vv

Once Script 2 has finished you'll need to edit /opt/zimbra/conf/localconfig.xml to update the following with the values from the localconfig.xml file on the server you are migrating from, you need put them in to your new server in order for LDAP to start successfully, otherwise it will not start at all as it doesn't know the old password.

I wanted to automate this process but my knowledge in bash scripting does not go far enough to open files and copy text out of them in specific places to paste them into different files in specific places , so you'll have to do this part by hand but its very easy!

You could essentially copy the file over but you would need to update the Zimbra GID as its going to mostly likely different, you can find it in the /etc/passwd file, but I digress.

Entries you need to update on the new servers localconfig.xml from the old localconfig.xml file on the source machine.
a. zimbra_mysql_password
b. mysql_root_password
c. zimbra_logger_mysql_password
d. mailboxd_keystore_password
e. mailboxd_truststore_password
f. mailboxd_keystore_base_password
g. zimbra_ldap_password
h. ldap_root_password
i. ldap_postfix_password
j. ldap_amavis_password
k. ldap_nginx_password
l. ldap_replication_password

Once you get the localconfig.xml updated with the existing LDAP password, run these commands

1. As root mv /opt/zimbra/mailboxd/etc/keystore /root/keystore.old
2. As root - # /opt/zimbra/bin/zmcertmgr deploycrt self
3. As root - /opt/zimbra/libexec/zmfixperms --verbose --extended
4. As root - /etc/init.d/zimbra restart

and your migration should be successful.

PLEASE POST ANY ERRORS, PROBLEMS, SUGGESTIONS, OR COMMENTS YOU HAVE!

Scripts you need to copy

Script 1 - Run as root on source server

#!/bin/bash

#Original script idea and creator - Eclipse
#Bug fixer - R0b3rt - Many thanks go to you R0b3rt!

#Colors!
COL_NORM="$(tput setaf 9)"
COL_RED="$(tput setaf 1)"
COL_GREEN="$(tput setaf 2)"
COL_BLUE="$(tput setaf 4)"

function pause(){
   read -p "$*"
}

ID=`id -u`

if [ "x$ID" != "x0" ]; then
	echo "You need to run this as root!"
	exit 1
fi

echo "${COL_RED}"
echo -n "Thank you for downloading and using my script, I do not guarantee nor do I "
echo "take responsibility for any lost or damaged information upon using this script, "
echo "please test this script on your own machines before using it."

echo ""

echo "I do not guarentee this script to work on every set up out there."
echo "This script is only for 1:1 migrations, it cannot migrate replicated or "
echo "external LDAP servers without modification."

echo ""

echo "In order for this script to run succesfully, please have both servers with"
echo "the exact same hostname, have rsync installed, and have the same version of"
echo "zimbra installed on the new machine properly setup running with out errors"
echo ""
echo "PLEASE FOLLOW AND ENSURE EVERYTHING YOU ENTER IS CORRECT WHEN ASKED "
echo "TO INPUT SOMETHING!"

while true
   do
     echo
       echo -n "$COL_NORM Would you like to continue [Y/N] ?${COL_NORM}"
       read continue
           case $continue in
            yes|Yes|YES|y|Y) break ;;
            no|No|n|N|NO)
       echo -n "${COL_RED} Good bye ${COL_NORM} "
       exit
     ;;
   *)
 esac
done

if [ $continue = "Y" -o $continue = "y" ] ; then

while true
   do
     echo
       echo -n "$COL_NORM Is the computer you're reading this on your existing mail server you are wanting to migrate from? [Y/N] ${COL_NORM} "
       read serversetup
           case $serversetup in
            yes|Yes|YES|y|Y) break ;;
            no|No|n|N|NO)
       echo -n "${COL_RED} Good bye ${COL_NORM} "
       exit
     ;;
   *)
 esac
done
	
fi

if [ $serversetup = "N" -o $serversetup = "n" ] ; then

	echo ""
	echo "Please run this on the source mail server you are wanting to migrate from."
	echo "Script exiting..."
	exit

elif [ $serversetup = "Y" -o $serversetup = "y" ] ; then
 
	echo "Creating backup directory"
	mkdir /ZIMMIGbackup
		
	echo "Creating permissions for our Zimbra user"
	chown zimbra:zimbra /ZIMMIGbackup  || { echo >&2 "There is something wrong with your Zimbra configuration stopping"
	echo "to prevent errors"; exit 1; }
	
	/etc/init.d/zimbra stop

	su - "zimbra" -c '/opt/zimbra/libexec/zmslapcat /ZIMMIGbackup'
	su - "zimbra" -c '/opt/zimbra/libexec/zmslapcat -c /ZIMMIGbackup'

	echo "Script complete, please run $COL_RED Script 2 $COL_NORM on the destination computer."
	exit
	#Finished inital setup.
fi 
	
#Fail safe
else
	echo "You've done something terribly wrong, exiting"
	exit

fi

Script 2 - Run as root on destination server

#! /bin/bash

#Original script idea and creator - Eclipse
#Bug fixer - R0b3rt - Many thanks go to you R0b3rt!

#Colors
COL_NORM="$(tput setaf 9)"
COL_RED="$(tput setaf 1)"
COL_GREEN="$(tput setaf 2)"
COL_BLUE="$(tput setaf 4)"

#function pause () {
#read -p "$*"
#}

ID=`id -u`

if [ "x$ID" != "x0" ]; then
	echo -n "${COL_RED} You need to run this as root! ${COL_NORM} "
exit 1
fi

echo ""
echo -n "${COL_RED} Thank you for downloading and using my script, I do not guarantee nor do I "
echo "take responsibility for any lost or damaged information upon using this script, "
echo "please test this script on your own machines before using it."

echo ""

echo "I do not guarentee this script to work on every set up out there."
echo "This script is only for 1:1 migrations, it cannot migrate replicated or "
echo "external LDAP servers without modification."

echo ""

echo "In order for this script to run succesfully, please have both servers with"
echo "the exact same hostname, have rsync installed, and have the same version of"
echo "zimbra installed on the new machine properly setup"
echo ""
echo "PLEASE FOLLOW AND ENSURE EVERYTHING YOU ENTER IS CORRECT WHEN ASKED "
echo "TO INPUT SOMETHING!"

while true
   do
     echo
       echo -n "$COL_NORM Would you like to continue  [Y/N] ?${COL_NORM} "
       read continue
           case $continue in
            yes|Yes|YES|y|Y) break ;;
            no|No|n|N|NO)
       echo -n "${COL_RED} Good bye ${COL_NORM} "
       exit
     ;;
   *)
 esac
done

if [[ $continue = [Yy] ]] ; then

       echo -n "${COL_RED} Is the computer your reading this on the destination server [Y/N] ? ${COL_NORM}"
       read    serversetup

fi

if [[ $continue = [Nn] ]] ; then

       echo -n "${COL_RED} Good bye. ${COL_NORM}"
       exit

fi

if [[ $serversetup = [Nn] ]] ; then

	echo "${COL_RED} Please run this on the destination server . ${COL_NORM} "
exit


elif [[ $serversetup = [Yy] ]] ; then 

	echo "${COL_BLUE} Starting inital preparation for the new server ${COL_NORM}"
#	pause "Press any key to continue"

	echo "${COL_BLUE} Creating backup directory ${COL_NORM}"
	mkdir /ZIMMIGbackup || { echo >&2 "There is something wrong with your Zimbra configuration stopping"
	echo "to prevent errors"; exit 1; }

	echo "${COL_BLUE} Creating permissions for our Zimbra user ${COL_NORM}"
	chown zimbra:zimbra /ZIMMIGbackup

	echo "${COL_BLUE} Shutting down the Zimbra services to begin migration job. ${COL_NORM}"

	/etc/init.d/zimbra stop

	echo "${COL_BLUE} Removing current LDAP data and preparing for the existing server ${COL_NORM}"
	rm -rfv /opt/zimbra/data/ldap/config/*
	rm -rfv /opt/zimbra/data/ldap/hdb/*
	rm -rfv /opt/zimbra/data/ldap/accesslog/* 
	mkdir -vp /opt/zimbra/data/ldap/hdb/db /opt/zimbra/data/ldap/hdb/logs 
	mkdir -vp /opt/zimbra/data/ldap/accesslog/db /opt/zimbra/data/ldap/accesslog/logs

	#Copying LDAP backup to new machine	begin

	echo "${COL_BLUE} Creating connection to old server to copy LDAP backup files  ${COL_NORM}"

echo -n "${COL_RED} Please enter the IP address or domain name of the server your migrating from: ${COL_NORM} "
read ipaddress
  while true
    do
      echo
	echo  -n "${COL_BLUE} You've entered: $ipaddress , is this correct? [Y/N]  ${COL_NORM}" 
        read iscorrect	
  	    case $iscorrect in
             yes|Yes|YES|y|Y) break ;;
             no|No|n|N|NO)
	echo -n "${COL_RED} Please enter the IP address or domain name of the server your migrating from: ${COL_NORM} "
	read ipaddress
      ;;
    *) 	
  esac
done
	
	echo ""

	echo "${COL_RED} Attempting to contact the server, you'll need to enter the root password when prompted  ${COL_NORM}"	

	echo "${COL_BLUE} Copying config files to the local /backup directory... ${COL_NORM}"
	rsync --progress -avrzH root@$ipaddress:/ZIMMIGbackup/* /ZIMMIGbackup

	#Copying LDAP backup to new machine	end	

	echo "${COL_BLUE} Finished copying LDAP files, starting import process... ${COL_NORM}"
	/opt/zimbra/openldap/sbin/slapadd -q -n 0 -F /opt/zimbra/data/ldap/config -cv -l /ZIMMIGbackup/ldap-config.bak
	echo "${COL_RED} Import process step 1/2 done. ${COL_NORM}"
	/opt/zimbra/openldap/sbin/slapadd -q -b "" -F /opt/zimbra/data/ldap/config -cv -l /ZIMMIGbackup/ldap.bak
	echo "${COL_RED} Import process step 2/2 done."

	echo "${COL_BLUE} Deleting current database ${COL_NORM}"
	rm -rf /opt/zimbra/db/data/*

    #Copying mysql DB from old machine to new BEGIN

	echo "${COL_RED} Creating connection to old server to copy existing database ${COL_NORM}"
	
	echo ""

	echo "${COL_BLUE} Attempting to contact the server, you'll need to enter the root password when prompted ${COL_NORM}"	

	echo "${COL_BLUE} Copying config files to the local /backup directory... ${COL_NORM}"
	rsync --progress -avrzH -e ssh root@$ipaddress:/opt/zimbra/db/data/* /opt/zimbra/db/data/	|| { echo >&2 "There is was a problem, stopping"
	echo "to prevent errors"; exit 1; }

    #Copying mysql DB from old machine to new END
    #Copying message store information from old machine to new BEGIN

	echo "${COL_BLUE} Creating connection to old server to copy existing mail store ${COL_NORM}"

	echo ""

	echo "${COL_BLUE} Attempting to contact the server, you'll need to enter the root password when prompted ${COL_NORM}"	

	echo "${COL_BLUE} Copying config files to the local /backup directory... ${COL_NORM}"

	rsync --progress -avrzH root@$ipaddress:/opt/zimbra/store/* /opt/zimbra/store/


    #Copying message store information from old machine to new END

	#Copying index information from old machine to new BEGIN

	echo "${COL_BLUE} Creating connection to old server to copy existing mail store ${COL_NORM}"

	echo ""

	echo "${COL_BLUE} Attempting to contact the server, you'll need to enter the root password when prompted ${COL_NORM}"	

	echo ""

	echo "${COL_BLUE} Copying config files to the local /backup directory... ${COL_NORM}"

	rsync --progress -avrzH root@$ipaddress:/opt/zimbra/index/*	 /opt/zimbra/index/	

    #Copying index information from old machine to new END

    #Removing old backup sessions that are no longer needed

	echo "${COL_BLUE} Removing old backup sessions ${COL_NORM}"
	rm -rf /opt/zimbra/redolog/*

	echo "${COL_BLUE} Rewriting permissions for the Zimbra account ${COL_NORM}"
	/opt/zimbra/libexec/zmfixperms --verbose --extended
	
#Finish
	echo "${COL_GREEN} Script complete $COL_NORM, please read and follow the rest of the steps on the Zimbra Migration Script thread ${COL_NORM}"
exit
fi
	fi

#Fail safe	
else

	echo "${COL_GREEN} You've done something very wrong, exiting. ${COL_NORM}"
exit
fi

Please leave suggestions, feed back, success stories, errors, and problems in replies

[andrussuitsu]

Many thanks for your post and the scripts. I have never got so far with a 32-bit to 64-bit migration.

At the moment I have still some issues left. When accessing my e-mails over the web interface I get mail.NO_SUCH_BLOB errors for some Folders. I will retry the rsync part once more when I get the chance to shut down the original Zimbra. Maybe the problems originates from the original 32-bit Zimbra server, so I will check the consistency of that just in case.

Still... There seems to be hope at the end of the tunnel!

[andrussuitsu]

I found out the cause of my errors. I ran script 2 against a running source mailserver first, then stopped it and re-ran the script again. But the script didn't have the --delete flag specified for rsync so mail messages that had already been deleted from the source server were left intact on the destination server. I modified the script to include the --delete flag, re-ran the script and got a working system on the destination mailserver!