Backup script didn't complete via crontab

[snake_eyes]

Hello,

After long time of tracing of my matter which is the backup script didn't complete via crontab (It export a zip file its size 500MB) unlike when I run it manual (It export a zip file its size 2.2 GB), I noticed that during execution in the terminal it show the following messages:

tar: zimbra/data/postfix/spool/public/cleanup: socket ignored
tar: zimbra/data/postfix/spool/public/flush: socket ignored
tar: zimbra/data/postfix/spool/public/showq: socket ignored
tar: zimbra/data/postfix/spool/private/ifmail: socket ignored
tar: zimbra/data/postfix/spool/private/error: socket ignored
tar: zimbra/data/postfix/spool/private/uucp: socket ignored
tar: zimbra/data/postfix/spool/private/proxymap: socket ignored
tar: zimbra/data/postfix/spool/private/trace: socket ignored
tar: zimbra/data/postfix/spool/private/maildrop: socket ignored
tar: zimbra/data/postfix/spool/private/verify: socket ignored
tar: zimbra/data/postfix/spool/private/smtp-amavis: socket ignored
tar: zimbra/data/postfix/spool/private/bsmtp: socket ignored
tar: zimbra/data/postfix/spool/private/discard: socket ignored
tar: zimbra/data/postfix/spool/private/cyrus: socket ignored
tar: zimbra/data/postfix/spool/private/bounce: socket ignored
tar: zimbra/data/postfix/spool/private/defer: socket ignored
tar: zimbra/data/postfix/spool/private/tlsmgr: socket ignored
tar: zimbra/data/postfix/spool/private/smtp: socket ignored
tar: zimbra/data/postfix/spool/private/relay: socket ignored
tar: zimbra/data/postfix/spool/private/rewrite: socket ignored
tar: zimbra/data/postfix/spool/private/lmtp: socket ignored
tar: zimbra/data/postfix/spool/private/anvil: socket ignored
tar: zimbra/data/postfix/spool/private/local: socket ignored
tar: zimbra/data/postfix/spool/private/virtual: socket ignored
tar: zimbra/data/postfix/spool/private/scache: socket ignored
tar: zimbra/data/postfix/spool/private/old-cyrus: socket ignored

I think because it show the socket ignored messages it didn't complete, in this case please advise if what I said is true otherwise how I can solve the above issue?

Here is the crontab command:

30 16 * * * /usr/src/backup/ZimbraBackup.sh 2>&1 >> /var/log/crontab_zimbra.log

Cheers,

[uxbod]

Those errors can be safely ignored. Sockets would be created when ZCS is restarted anyway. Are you error checking in the script ? And if so at what point does it return a error ?

[snake_eyes]

Hello uxbod,

Really I'm tired from this matter, look when I execute it manually from the terminal it completed as I mentioned with the tar.gz file 2.2 GB unlike when I execute it via crontab it export and tar.gz with 500 MB approximately,

Here is my script:

#!/bin/bash

# Global settings
local_backup_dir=/opt/backup/Zimbra
remote_backup_dir=/home/mailuser/appprod/backup/Zimbra
backup_full_file_name=ZimbraBackupFull_`date +%Y-%m-%d.%I-%M-%S-%p`.tar.gz
log_file_name=ZimbraBackup_`date +%Y-%m-%d.%I-%M-%S-%p`.log
log_file_name_local_path=$local_backup_dir/01/$log_file_name
log_file_name_remote_path=$remote_backup_dir/01/$log_file_name

# Create log file if not exist
touch $log_file_name_local_path

# Time backup started
echo "`date +%Y%m%d%H%M%S`, Time backup started: $(date +%T)"
echo "`date +%Y%m%d%H%M%S`, Time backup started: $(date +%T)" >> $log_file_name_local_path

# Waiting message
echo "`date +%Y%m%d%H%M%S`, Please Wait while preparing system for backup..."
before="$(date +%s)"

# Archiving backups
rm -rf $local_backup_dir/07
mv $local_backup_dir/06 $local_backup_dir/07
mv $local_backup_dir/05 $local_backup_dir/06
mv $local_backup_dir/04 $local_backup_dir/05
mv $local_backup_dir/03 $local_backup_dir/04
mv $local_backup_dir/02 $local_backup_dir/03
mv $local_backup_dir/01 $local_backup_dir/02
mkdir $local_backup_dir/01

rm -rf $remote_backup_dir/07
mv $remote_backup_dir/06 $remote_backup_dir/07
mv $remote_backup_dir/05 $remote_backup_dir/06
mv $remote_backup_dir/04 $remote_backup_dir/05
mv $remote_backup_dir/03 $remote_backup_dir/04
mv $remote_backup_dir/02 $remote_backup_dir/03
mv $remote_backup_dir/01 $remote_backup_dir/02
mkdir $remote_backup_dir/01

# Backup started
echo "`date +%Y%m%d%H%M%S`, Backup Started."
echo "`date +%Y%m%d%H%M%S`, Backup Started." >> $log_file_name_local_path

# Check which zimbra version that is installed, for recovery purpose
echo "`date +%Y%m%d%H%M%S`, Check which zimbra version that is installed."
echo "`date +%Y%m%d%H%M%S`, Check which zimbra version that is installed." >> $log_file_name_local_path
su - zimbra -c "zmcontrol -v" >> $log_file_name_local_path

# Creating required directories
echo "`date +%Y%m%d%H%M%S`, Creating required directories"
echo "`date +%Y%m%d%H%M%S`, Creating required directories" >> $log_file_name_local_path
mkdir -p $local_backup_dir
mkdir -p $remote_backup_dir
mkdir -p /opt/backup/tmp
rm -rf /opt/backup/tmp/zimbra

# Now we need to shut down Zimbra to rsync any files that were/are locked
before2="$(date +%s)"

# Zimbra services stopping
echo "`date +%Y%m%d%H%M%S`, Zimbra services stopping..."
echo "`date +%Y%m%d%H%M%S`, Zimbra services stopping..." >> $log_file_name_local_path
su - zimbra -c "zmcontrol stop" >> $log_file_name_local_path

sleep 25

echo "`date +%Y%m%d%H%M%S`, Zimbra services stopped."
echo "`date +%Y%m%d%H%M%S`, Zimbra services stopped." >> $log_file_name_local_path

# Getting zimbra procecees
echo "`date +%Y%m%d%H%M%S`, Getting zimbra procecees"
echo "`date +%Y%m%d%H%M%S`, Getting zimbra procecees" >> $log_file_name_local_path
ps aux | grep zimbra >> $log_file_name_local_path

# Killing zimbra procecees
echo "`date +%Y%m%d%H%M%S`, Killing zimbra procecees"
echo "`date +%Y%m%d%H%M%S`, Killing zimbra procecees" >> $log_file_name_local_path
pkill -9 -u zimbra >> $log_file_name_local_path

# Gold syncing to backup folder
echo "`date +%Y%m%d%H%M%S`, Gold syncing to backup folder..."
echo "`date +%Y%m%d%H%M%S`, Gold syncing to backup folder..." >> $log_file_name_local_path
rsync -avr /opt/zimbra/ /opt/backup/tmp/zimbra >> $log_file_name_local_path

# Compressing backup for space reduction
echo "`date +%Y%m%d%H%M%S`, Compressing backup for space reduction..."
echo "`date +%Y%m%d%H%M%S`, Compressing backup for space reduction..." >> $log_file_name_local_path
tar -zcvf $local_backup_dir/01/$backup_full_file_name -C /opt/backup/tmp zimbra >> $log_file_name_local_path
#cd /opt/backup/tmp
#tar zcvf $local_backup_dir/01/$backup_full_file_name zimbra 2&> /dev/null >> $log_file_name_local_path
echo "`date +%Y%m%d%H%M%S`, Compressed backup folder."
echo "`date +%Y%m%d%H%M%S`, Compressed backup folder." >> $log_file_name_local_path

# Zimbra services starting
echo "`date +%Y%m%d%H%M%S`, Zimbra services starting..."
echo "`date +%Y%m%d%H%M%S`, Zimbra services starting..." >> $log_file_name_local_path
su - zimbra -c "zmcontrol start" >> $log_file_name_local_path

sleep 25

echo "`date +%Y%m%d%H%M%S`, Zimbra services started."
echo "`date +%Y%m%d%H%M%S`, Zimbra services started." >> $log_file_name_local_path

# Displaying Zimbra services status
echo "`date +%Y%m%d%H%M%S`, Displaying Zimbra services status"
echo "`date +%Y%m%d%H%M%S`, Displaying Zimbra services status" >> $log_file_name_local_path
su - zimbra -c "zmcontrol status" >> $log_file_name_local_path

# Calculates and outputs amount of time the server was down for
after="$(date +%s)"
elapsed="$(expr $after - $before2)"
hours=$(($elapsed / 3600))
elapsed=$(($elapsed - $hours * 3600))
minutes=$(($elapsed / 60))
seconds=$(($elapsed - $minutes * 60))
echo "`date +%Y%m%d%H%M%S`, Server was down for: $hours hours $minutes minutes $seconds seconds"
echo "`date +%Y%m%d%H%M%S`, Server was down for: $hours hours $minutes minutes $seconds seconds" >> $log_file_name_local_path

# Outputs the time the backup finished
echo "`date +%Y%m%d%H%M%S`, Time backup finished: $(date +%T)"
echo "`date +%Y%m%d%H%M%S`, Time backup finished: $(date +%T)" >> $log_file_name_local_path

# Calculates and outputs total time taken
after="$(date +%s)"
elapsed="$(expr $after - $before)"
hours=$(($elapsed / 3600))
elapsed=$(($elapsed - $hours * 3600))
minutes=$(($elapsed / 60))
seconds=$(($elapsed - $minutes * 60))
echo "`date +%Y%m%d%H%M%S`, Time taken: $hours hours $minutes minutes $seconds seconds"
echo "`date +%Y%m%d%H%M%S`, Time taken: $hours hours $minutes minutes $seconds seconds" >> $log_file_name_local_path

# Backup finished
echo "`date +%Y%m%d%H%M%S`, Backup finished."
echo "`date +%Y%m%d%H%M%S`, Backup finished." >> $log_file_name_local_path

# Moving backup files
echo "`date +%Y%m%d%H%M%S`, Moving $local_backup_dir/01/$backup_full_file_name to $remote_backup_dir/01/"
echo "`date +%Y%m%d%H%M%S`, Moving $local_backup_dir/01/$backup_full_file_name to $remote_backup_dir/01/" >> $log_file_name_local_path
cp $log_file_name_local_path $log_file_name_remote_path

echo "`date +%Y%m%d%H%M%S`, Moved successfully."
echo "`date +%Y%m%d%H%M%S`, Moved successfully." >> $log_file_name_local_path
cp $local_backup_dir/01/$backup_full_file_name $remote_backup_dir/01/$backup_full_name

and here is my crontab jobs;

11 21 * * * /etc/webmin/cron/tempdelete.pl
30 16 * * * /usr/src/backup/ZimbraBackup.sh 2>&1 >> /var/log/crontab_zimbra.log

Here is the crontab_zimbra.log

20090530163001, Time backup started: 16:30:01
20090530163001, Please Wait while preparing system for backup...
20090530163004, Backup Started.
20090530163004, Check which zimbra version that is installed.
20090530163011, Creating directories
20090530163017, Zimbra services stopping...
20090530163114, Zimbra services stopped.
20090530163114, Getting zimbra procecees
20090530163114, Killing zimbra procecees
20090530163114, Gold syncing to backup folder...
20090530163830, Compressing backup for space reduction...
20090530164034, Compressed backup folder.
20090530164034, Zimbra services starting...
20090530164234, Zimbra services started.
20090530164234, Displaying Zimbra services status
20090530164238, Server was down for: 0 hours 12 minutes 21 seconds
20090530164238, Time backup finished: 16:42:38
20090530164238, Time taken: 0 hours 12 minutes 37 seconds
20090530164238, Backup finished.
20090530164238, Moving /opt/backup/Zimbra/01/ZimbraBackupFull_2009-05-30.04-30-01-PM.tar.gz to /home/mailuser/appprod/backup/Zimbra/01/
20090530164239, Moved successfully.

Please note: it took 12 minutes via crontab and 20 minutes manually, so what's going on?

Cheers,

[snake_eyes]

Any suggest gentleman?

[uxbod]

Can you post the output of ZimbraBackupFull.${DATE} ?

[snake_eyes]

Yes sure...

Check this link out: http://ubuntuway.net/ZimbraBackup_20...4-30-01-PM.log

Cheers,

[uxbod]

And the interactive one ? That way we can compare what is happening differently between the two.

[snake_eyes]

Here is the completed task via terminal which is export tar.gz 2.2GB

http://ubuntuway.net/ZimbraBackup_20...4-30-01-PM.log

Cheers,