[SOLVED] Yet Another Backup Script for Community Version

[phoenix]

Originally Posted by webweaver

Thanks Heinzg,
I'm using a CentOS 5.1 box and I really expected yum to go get DAR for me but it didn't. I'm not positive which repositories are setup on the box at the moment, perhaps some other one would have Dar than a stock build of Cent? I know enough linux to get myself in trouble, lol, but I can follow directions easily. When ./configure fails I am stumped.

Dave

You'll find DAR on the RPMforge repository
.

[RaNd]

Originally Posted by RaNd

Can DAR files be viewed by like tar.gz from Midnight Commander ?
I haven't use dar files before, so here comes a simple (dummy) question :
How can i restore the full and/or the diff backup ??

I've got also error in line 901 , I'll try to add someone to CC to check . .
Ubuntu 6.06.2
Zimbra FOSS 5.05


PS Great Script!!!

Update
Added an email to EMAILCC cause it was messing up the outgoing report.

Create file listing from archive

:::::::::::::::::::::::::::::::::::::::::::::::::: ::::::::::::::::::::
Full Zimbra Backup ended at: 18:55
Backup took Hr:0 Min:7 Sec:4 to complete
:::::::::::::::::Cheers Osoffice for the script:::::::::::::::::::::::
./zmbac.sh: line 901: /backup/zimbra_dars/Backuplist-07-May-2008.txt.gz: Permission denied
Null message body; hope that's ok

On a clean system, clean install . . .

[ebariaux]

On CentOS 5, make sure to do yum install gcc-c++ before compiling dar, this should solve your size_t no found issue.

Eric

[chrishewitt]

Originally Posted by heinzg

As I can remember it is not a bug... I want a weekly log with all info i.e. the logging of both Diff & Full backups

Sorry Heinz.. I made some wrong changes to crontab and things got a bit messed up so I had to wipe everything and start over. I've since put crontab back to the example used in the script and disabled encryption. I made the 1st full backup manually (on the same day that crontab would have run a full backup) with subsequent diff backups run normally from crontab. The manual full backup didn't generate a log file - although I deleted the original file and touched a new one as part of the cleanup process, and that might be why. The null/empty log file was emailed normally though. Without encryption enabled the diff backups appear to create the Backuplist-<date>.txt.gz files okay without throwing the line 901/elastic buffer error. I have someone in the CC list for the email section of the script but neither address receives an email confirming diff backup completion and the log file shows a cumulative list of diff backups for the week, not just the latest diff backup event, so the log is not being cleared/reset on the start of the next diff backup run, which is the behavior I would expect since a diff backup appears to reset the log file (I could be wrong here.. like I said things got a bit screwy).

NB: I built dar as per the simple instructions on the project's website and using the sources from the same site. I get the following running dar -V

dar version 2.3.7, Copyright (C) 2002-2052 Denis Corbin
Long options support : YES

Using libdar 4.4.2 built with compilation time options:
Libz compression (gzip) : YES
Libbz2 compression (bzip2) : YES
Strong encryption : YES
New Blowfish implementation: YES
Extended Attributes support: YES
Large files support (> 2GB): YES
ext2fs NODUMP flag support : YES
Special allocation scheme : YES
Integer size used : unlimited
Thread safe support : YES

compiled the Apr 20 2008 with GNUC version 4.1.2 20070626 (Red Hat 4.1.2-14)
dar is part of the Disk ARchive suite (Release 2.3.7)

I'm not sure if that's relevant or useful, but hey. Please reply or PM me if there's any specific information you need. Based on what other people have posted as script output, the only information of relevance for the line 901 error is what I already posted - everything else in the logs is normal.

Final comment re the unusual elapsed time counter. I've noticed that the Sun NFS box I'm using as the backup target isn't using NTP and the clock is about 5 minutes ahead. I'm not sure if that's a contributing factor in the 1hr 84 mins elapsed time I see recorded on a full backup?

Christian

[chrishewitt]

Hmmmm... I had the first "end of week" event overnight and things rolled over and I got the following email this morning. Any ideas on why the Java exception is thrown?

>>>>>>>>>>> START <<<<<<<<<<<<<

Full Backup started at: 01:30

Old backups found...old week= 18 current week= 19

/opt/nfs/zimbra-backup-current/18_Zimbra_Backup_06-May-2008_FULL.1.dar moved to /opt/nfs/zimbra-backup-old/
/opt/nfs/zimbra-backup-current/18_Zimbra_Backup_06-May-2008_FULL.1.dar.md5 moved to /opt/nfs/zimbra-backup-old/
/opt/nfs/zimbra-backup-current/18_Zimbra_Backup_07-May-2008_DIFF.1.dar moved to /opt/nfs/zimbra-backup-old/
/opt/nfs/zimbra-backup-current/18_Zimbra_Backup_07-May-2008_DIFF.1.dar.md5 moved to /opt/nfs/zimbra-backup-old/
/opt/nfs/zimbra-backup-current/18_Zimbra_Backup_08-May-2008_DIFF.1.dar moved to /opt/nfs/zimbra-backup-old/
/opt/nfs/zimbra-backup-current/18_Zimbra_Backup_08-May-2008_DIFF.1.dar.md5 moved to /opt/nfs/zimbra-backup-old/
/opt/nfs/zimbra-backup-current/18_Zimbra_Backup_09-May-2008_DIFF.1.dar moved to /opt/nfs/zimbra-backup-old/
/opt/nfs/zimbra-backup-current/18_Zimbra_Backup_09-May-2008_DIFF.1.dar.md5 moved to /opt/nfs/zimbra-backup-old/
/opt/nfs/zimbra-backup-current/18_Zimbra_Backup_10-May-2008_DIFF.1.dar moved to /opt/nfs/zimbra-backup-old/
/opt/nfs/zimbra-backup-current/18_Zimbra_Backup_10-May-2008_DIFF.1.dar.md5 moved to /opt/nfs/zimbra-backup-old/
/opt/nfs/zimbra-backup-current/18_Zimbra_Backup_11-May-2008_DIFF.1.dar moved to /opt/nfs/zimbra-backup-old/
/opt/nfs/zimbra-backup-current/18_Zimbra_Backup_11-May-2008_DIFF.1.dar.md5 moved to /opt/nfs/zimbra-backup-old/
java.net.UnknownHostException: zim.chrishewitt.net: zim.chrishewitt.net
at java.net.InetAddress.getLocalHost(InetAddress.java :1353)
at com.zimbra.common.localconfig.LC.<clinit>(LC.java: 350)
at com.zimbra.common.localconfig.KnownKey.<clinit>(Kn ownKey.java:32)
at com.zimbra.common.localconfig.LocalConfig.expandAl l(LocalConfig.java:327)
at com.zimbra.common.localconfig.LocalConfig.<init>(L ocalConfig.java:177)
at com.zimbra.common.localconfig.LocalConfig.readConf ig(LocalConfig.java:343)
at com.zimbra.common.localconfig.LocalConfig.<clinit> (LocalConfig.java:348)
at com.zimbra.common.localconfig.Main.main(Main.java: 108)

Setting date & version Marker into /opt/zimbra/DATE_VERSION.txt

java.net.UnknownHostException: zim.chrishewitt.net: zim.chrishewitt.net
at java.net.InetAddress.getLocalHost(InetAddress.java :1353)
at com.zimbra.common.localconfig.LC.<clinit>(LC.java: 350)
at com.zimbra.common.localconfig.KnownKey.<clinit>(Kn ownKey.java:32)
at com.zimbra.common.localconfig.LocalConfig.expandAl l(LocalConfig.java:327)
at com.zimbra.common.localconfig.LocalConfig.<init>(L ocalConfig.java:177)
at com.zimbra.common.localconfig.LocalConfig.readConf ig(LocalConfig.java:343)
at com.zimbra.common.localconfig.LocalConfig.<clinit> (LocalConfig.java:348)
at com.zimbra.common.localconfig.Main.main(Main.java: 108)
Doing a hotsync of /opt/zimbra/ to /tmp/zimbra-backup-tmp/
sync went ok

Stopping the Zimbra server

Deactivated zimbra's cronjobs

java.net.UnknownHostException: zim.chrishewitt.net: zim.chrishewitt.net
at java.net.InetAddress.getLocalHost(InetAddress.java :1353)
at com.zimbra.common.localconfig.LC.<clinit>(LC.java: 350)
at com.zimbra.common.localconfig.KnownKey.<clinit>(Kn ownKey.java:32)
at com.zimbra.common.localconfig.LocalConfig.expandAl l(LocalConfig.java:327)
at com.zimbra.common.localconfig.LocalConfig.<init>(L ocalConfig.java:177)
at com.zimbra.common.localconfig.LocalConfig.readConf ig(LocalConfig.java:343)
at com.zimbra.common.localconfig.LocalConfig.<clinit> (LocalConfig.java:348)
at com.zimbra.common.localconfig.Main.main(Main.java: 108)
Host zim.chrishewitt.net
Stopping stats...Done
Stopping mta...Done
Stopping spell...Done
Stopping snmp...Done
Stopping archiving...Done
Stopping antivirus...Done
Stopping antispam...Done
Stopping imapproxy...Done
Stopping mailbox...Done
Stopping logger...Done
Stopping ldap...Done

Zimbra server has stopped

kill any left over zimbra procs
Did a Mr. Bush on some left over procs...

Doing a fast cold sync...
sync went ok

Reinstated zimbra's cronjobs

java.net.UnknownHostException: zim.chrishewitt.net: zim.chrishewitt.net
at java.net.InetAddress.getLocalHost(InetAddress.java :1353)
at com.zimbra.common.localconfig.LC.<clinit>(LC.java: 350)
at com.zimbra.common.localconfig.KnownKey.<clinit>(Kn ownKey.java:32)
at com.zimbra.common.localconfig.LocalConfig.expandAl l(LocalConfig.java:327)
at com.zimbra.common.localconfig.LocalConfig.<init>(L ocalConfig.java:177)
at com.zimbra.common.localconfig.LocalConfig.readConf ig(LocalConfig.java:343)
at com.zimbra.common.localconfig.LocalConfig.<clinit> (LocalConfig.java:348)
at com.zimbra.common.localconfig.Main.main(Main.java: 108)
Host zim.chrishewitt.net
Starting ldap...Done.
Starting logger...Done.
Starting mailbox...Done.
Starting snmp...Done.
Starting spell...Done.
Starting mta...Done.
Starting stats...Done.

Zimbra has started

Service down time was - Hr:0 Min:10 Sec:34

Running A hack... This one to check and start Stats subsystem
Hack not needed Stats seems to be running fine...

All services are running


Writing a full backup called:
19_Zimbra_Backup_12-May-2008_FULL
in: /opt/nfs/zimbra-backup-current/ with file sizes of max: 4395M
Archive is NOT Encrypted!!!


--------------------------------------------
43236 inode(s) saved
with 1124 hard link(s) recorded
0 inode(s) changed at the moment of the backup
0 inode(s) not saved (no inode/file change)
0 inode(s) failed to save (filesystem error)
0 inode(s) ignored (excluded by filters)
0 inode(s) recorded as deleted from reference backup
--------------------------------------------
Total number of inode considered: 43236
--------------------------------------------
EA saved for 0 inode(s)
--------------------------------------------
Archive Created successfully

Create MD5 Checksum for 19_Zimbra_Backup_12-May-2008_FULL.1.dar

MD5 Checksum Created successfully


Create file listing from archive

:::::::::::::::::::::::::::::::::::::::::::::::::: :::::::::::::::::::
Full Zimbra Backup ended at: 03:08
Backup took Hr:1 Min:98 Sec:31 to complete
:::::::::::::::::Cheers Osoffice for the script:::::::::::::::::::::::

>>>>>>>>>>>>> END <<<<<<<<<<<<

Note the strange elapsed time (as normal)..

Any ideas?

Christian

[heinzg]

Originally Posted by chrishewitt

Hmmmm... I had the first "end of week" event overnight and things rolled over and I got the following email this morning. Any ideas on why the Java exception is thrown?

>>>>>>>>>>> START <<<<<<<<<<<<<

Full Backup started at: 01:30

Old backups found...old week= 18 current week= 19
......
java.net.UnknownHostException: zim.chrishewitt.net: zim.chrishewitt.net
at java.net.InetAddress.getLocalHost(InetAddress.java :1353)
at com.zimbra.common.localconfig.LC.<clinit>(LC.java: 350)
at com.zimbra.common.localconfig.KnownKey.<clinit>(Kn ownKey.java:32)
at com.zimbra.common.localconfig.LocalConfig.expandAl l(LocalConfig.java:327)
at com.zimbra.common.localconfig.LocalConfig.<init>(L ocalConfig.java:177)
at com.zimbra.common.localconfig.LocalConfig.readConf ig(LocalConfig.java:343)
at com.zimbra.common.localconfig.LocalConfig.<clinit> (LocalConfig.java:348)
at com.zimbra.common.localconfig.Main.main(Main.java: 108)

Setting date & version Marker into /opt/zimbra/DATE_VERSION.txt

java.net.UnknownHostException: zim.chrishewitt.net: zim.chrishewitt.net
at java.net.InetAddress.getLocalHost(InetAddress.java :1353)
at com.zimbra.common.localconfig.LC.<clinit>(LC.java: 350)
at com.zimbra.common.localconfig.KnownKey.<clinit>(Kn ownKey.java:32)
at com.zimbra.common.localconfig.LocalConfig.expandAl l(LocalConfig.java:327)
at com.zimbra.common.localconfig.LocalConfig.<init>(L ocalConfig.java:177)
at com.zimbra.common.localconfig.LocalConfig.readConf ig(LocalConfig.java:343)
at com.zimbra.common.localconfig.LocalConfig.<clinit> (LocalConfig.java:348)
at com.zimbra.common.localconfig.Main.main(Main.java: 108)

......


:::::::::::::::::::::::::::::::::::::::::::::::::: :::::::::::::::::::
Full Zimbra Backup ended at: 03:08
Backup took Hr:1 Min:98 Sec:31 to complete
:::::::::::::::::Cheers Osoffice for the script:::::::::::::::::::::::

>>>>>>>>>>>>> END <<<<<<<<<<<<

Note the strange elapsed time (as normal)..

Any ideas?

Christian

Hi Christian!

I have no idea why the Java exception is thrown. But it seems to happen when one calls 'zmcontrol' commands.
Try running a "zmcontrol status" as the zimbra user. I bet you will get the same exceptions.

About the funny time issue, I will have a look at that soon. I see the time stats a as a nice extra but not of real importance...( thus not that high on the list to fix in the limited time I currently have, sorry)

heinzg

[heinzg]

Originally Posted by chrishewitt

# cat Backuplist-24-April-2008.txt
FATAL error, aborting operation
too large elastic buffer or elastic buffer incoherent structure

Any ideas?

Christian

Hi Christian,

have close look at the following two commands they show K & J options with crypt FULL & DIFF files and if the options are used on the wrong archive you will get the error above.

dar -K bf:MYLONGCRYPTKEY -l 19_Zimbra_Backup_13-May-2008_FULL  | head
Warning, the archive 19_Zimbra_Backup_13-May-2008_FULL has been encrypted. A wrong key is not possible to detect, it would cause DAR to report the archive as corrupted
[data ][ EA  ][compr] | permission | user  | group | size  |          date                 |    filename
----------------------+------------+-------+-------+-------+-------------------------------+------------
[Saved]       [-----]   drwxr-xr-x   1001	zimbra	0	Tue May 13 20:16:30 2008	db
[Saved]       [-----]   drwxr-x---   zimbra	zimbra	0	Tue May 13 19:57:26 2008	db/data
[Saved]       [-----]   drwx------   zimbra	zimbra	0	Sun Mar 23 16:39:41 2008	db/data/test
[Saved]       [-----]   drwx------   zimbra	zimbra	0	Sun Mar 23 16:39:42 2008	db/data/mysql
[Saved]       [     ]   -rw-rw----   zimbra	zimbra	0	Sun Mar 23 16:39:42 2008	db/data/mysql/procs_priv.MYD
[Saved]       [  82%]   -rw-rw----   zimbra	zimbra	1024	Sun Mar 23 16:39:42 2008	db/data/mysql/procs_priv.MYI
[Saved]       [  94%]   -rw-rw----   zimbra	zimbra	8875	Sun Mar 23 16:39:42 2008	db/data/mysql/procs_priv.frm

and

dar -J bf:MYLONGCRYPTKEY -l 19_Zimbra_Backup_13-May-2008_DIFF | head
-J is only useful with -A option, for the archive of reference
[data ][ EA  ][compr] | permission | user  | group | size  |          date                 |    filename
----------------------+------------+-------+-------+-------+-------------------------------+------------
[Saved]       [-----]   drwxr-xr-x   1001	zimbra	0	Tue May 13 21:40:48 2008	db
[Saved]       [-----]   drwxr-x---   zimbra	zimbra	0	Tue May 13 20:18:22 2008	db/data
[     ]       [-----]   drwx------   zimbra	zimbra	0	Sun Mar 23 16:39:41 2008	db/data/test
[     ]       [-----]   drwx------   zimbra	zimbra	0	Sun Mar 23 16:39:42 2008	db/data/mysql
[     ]       [-----]   -rw-rw----   zimbra	zimbra	0	Sun Mar 23 16:39:42 2008	db/data/mysql/procs_priv.MYD
[     ]       [-----]   -rw-rw----   zimbra	zimbra	1024	Sun Mar 23 16:39:42 2008	db/data/mysql/procs_priv.MYI
[     ]       [-----]   -rw-rw----   zimbra	zimbra	8875	Sun Mar 23 16:39:42 2008	db/data/mysql/procs_priv.frm
[     ]       [-----]   -rw-rw----   zimbra	zimbra	77815	Sun Mar 23 16:39:42 2008	db/data/mysql/help_keyword.MYD

Now the error:

dar -K bf:MYLONGCRYPTKEY -l 19_Zimbra_Backup_13-May-2008_DIFF | head
FATAL error, aborting operation
too large elastic buffer or elastic buffer incoherent structure

Does this help you, Please check this agains your dar files, and let me know. That is if you still have some dar that are crypt.

heinzg

[gilus]

Hi,

First of all: GREAT TOOL! Thanks for sharing!

Unfortunately I' finding in the log files lines like this:
file has vanished: "/opt/zimbra/store/0/30/msg/8/34938-143841.msg"

Is it rsync error?
(Ububntu 6.06 LTS, Zimbra 5)

Thanks for any ideas,
Michal

[heinzg]

Originally Posted by gilus

Hi,

First of all: GREAT TOOL! Thanks for sharing!

Unfortunately I' finding in the log files lines like this:
file has vanished: "/opt/zimbra/store/0/30/msg/8/34938-143841.msg"

Is it rsync error?
(Ububntu 6.06 LTS, Zimbra 5)

Thanks for any ideas,
Michal

Hi Michal!

Cheers for the thanx!

Your question: Nope it is not to worry about.
This happens during hot syncs. in your case the email "34938-143841.msg" was deleted during hot sync. How it works is when rsync starts it builds an index of all the files it is going to transfer, and after that another of the transfered file which should match. But due to the file being delete between indexes it must of vanished .

But you should never have this during cold sync i.e. when the server is stopped to sync again.

Hope that helps.

Heinzg

[gilus]

Hi Heinzg,

Thanks again for fast reply! Indeed it happens only while hotsync (I guess it was because some of the users were still online and working! ;-) Cold sync went without any errors.

Again than You for a great tool! The people like You are the core of the Open Source idea! Thanks for the support and good luck!

Michal