Page 7 of 66 FirstFirst ... 567891757 ... LastLast
Results 61 to 70 of 659

Thread: [SOLVED] Yet Another Backup Script for Community Version

  1. #61
    phoenix is offline Zimbra Consultant & Moderator
    Join Date
    Sep 2005
    Location
    Vannes, France
    Posts
    23,201
    Rep Power
    56

    Default

    Quote Originally Posted by webweaver View Post
    Thanks Heinzg,
    I'm using a CentOS 5.1 box and I really expected yum to go get DAR for me but it didn't. I'm not positive which repositories are setup on the box at the moment, perhaps some other one would have Dar than a stock build of Cent? I know enough linux to get myself in trouble, lol, but I can follow directions easily. When ./configure fails I am stumped.

    Dave
    You'll find DAR on the RPMforge repository
    .
    Regards


    Bill


    Acompli: A new adventure for Co-Founder KevinH.

  2. #62
    RaNd is offline Active Member
    Join Date
    Feb 2007
    Location
    Greece
    Posts
    27
    Rep Power
    8

    Default

    Quote Originally Posted by RaNd View Post
    Can DAR files be viewed by like tar.gz from Midnight Commander ?
    I haven't use dar files before, so here comes a simple (dummy) question :
    How can i restore the full and/or the diff backup ??

    I've got also error in line 901 , I'll try to add someone to CC to check . .
    Ubuntu 6.06.2
    Zimbra FOSS 5.05


    PS Great Script!!!

    Update
    Added an email to EMAILCC cause it was messing up the outgoing report.

    Create file listing from archive

    :::::::::::::::::::::::::::::::::::::::::::::::::: ::::::::::::::::::::
    Full Zimbra Backup ended at: 18:55
    Backup took Hr:0 Min:7 Sec:4 to complete
    :::::::::::::::::Cheers Osoffice for the script:::::::::::::::::::::::
    ./zmbac.sh: line 901: /backup/zimbra_dars/Backuplist-07-May-2008.txt.gz: Permission denied
    Null message body; hope that's ok


    On a clean system, clean install . . .

  3. #63
    ebariaux is offline Starter Member
    Join Date
    May 2008
    Posts
    1
    Rep Power
    6

    Default

    On CentOS 5, make sure to do yum install gcc-c++ before compiling dar, this should solve your size_t no found issue.

    Eric

  4. #64
    chrishewitt's Avatar
    chrishewitt is offline Active Member
    Join Date
    Apr 2008
    Location
    Dubai, UAE
    Posts
    25
    Rep Power
    7

    Default

    Quote Originally Posted by heinzg View Post
    As I can remember it is not a bug... I want a weekly log with all info i.e. the logging of both Diff & Full backups
    Sorry Heinz.. I made some wrong changes to crontab and things got a bit messed up so I had to wipe everything and start over. I've since put crontab back to the example used in the script and disabled encryption. I made the 1st full backup manually (on the same day that crontab would have run a full backup) with subsequent diff backups run normally from crontab. The manual full backup didn't generate a log file - although I deleted the original file and touched a new one as part of the cleanup process, and that might be why. The null/empty log file was emailed normally though. Without encryption enabled the diff backups appear to create the Backuplist-<date>.txt.gz files okay without throwing the line 901/elastic buffer error. I have someone in the CC list for the email section of the script but neither address receives an email confirming diff backup completion and the log file shows a cumulative list of diff backups for the week, not just the latest diff backup event, so the log is not being cleared/reset on the start of the next diff backup run, which is the behavior I would expect since a diff backup appears to reset the log file (I could be wrong here.. like I said things got a bit screwy).

    NB: I built dar as per the simple instructions on the project's website and using the sources from the same site. I get the following running dar -V

    dar version 2.3.7, Copyright (C) 2002-2052 Denis Corbin
    Long options support : YES

    Using libdar 4.4.2 built with compilation time options:
    Libz compression (gzip) : YES
    Libbz2 compression (bzip2) : YES
    Strong encryption : YES
    New Blowfish implementation: YES
    Extended Attributes support: YES
    Large files support (> 2GB): YES
    ext2fs NODUMP flag support : YES
    Special allocation scheme : YES
    Integer size used : unlimited
    Thread safe support : YES

    compiled the Apr 20 2008 with GNUC version 4.1.2 20070626 (Red Hat 4.1.2-14)
    dar is part of the Disk ARchive suite (Release 2.3.7)

    I'm not sure if that's relevant or useful, but hey. Please reply or PM me if there's any specific information you need. Based on what other people have posted as script output, the only information of relevance for the line 901 error is what I already posted - everything else in the logs is normal.

    Final comment re the unusual elapsed time counter. I've noticed that the Sun NFS box I'm using as the backup target isn't using NTP and the clock is about 5 minutes ahead. I'm not sure if that's a contributing factor in the 1hr 84 mins elapsed time I see recorded on a full backup?

    Christian
    Last edited by chrishewitt; 05-11-2008 at 05:56 AM.

  5. #65
    chrishewitt's Avatar
    chrishewitt is offline Active Member
    Join Date
    Apr 2008
    Location
    Dubai, UAE
    Posts
    25
    Rep Power
    7

    Default

    Hmmmm... I had the first "end of week" event overnight and things rolled over and I got the following email this morning. Any ideas on why the Java exception is thrown?

    >>>>>>>>>>> START <<<<<<<<<<<<<

    Full Backup started at: 01:30

    Old backups found...old week= 18 current week= 19

    /opt/nfs/zimbra-backup-current/18_Zimbra_Backup_06-May-2008_FULL.1.dar moved to /opt/nfs/zimbra-backup-old/
    /opt/nfs/zimbra-backup-current/18_Zimbra_Backup_06-May-2008_FULL.1.dar.md5 moved to /opt/nfs/zimbra-backup-old/
    /opt/nfs/zimbra-backup-current/18_Zimbra_Backup_07-May-2008_DIFF.1.dar moved to /opt/nfs/zimbra-backup-old/
    /opt/nfs/zimbra-backup-current/18_Zimbra_Backup_07-May-2008_DIFF.1.dar.md5 moved to /opt/nfs/zimbra-backup-old/
    /opt/nfs/zimbra-backup-current/18_Zimbra_Backup_08-May-2008_DIFF.1.dar moved to /opt/nfs/zimbra-backup-old/
    /opt/nfs/zimbra-backup-current/18_Zimbra_Backup_08-May-2008_DIFF.1.dar.md5 moved to /opt/nfs/zimbra-backup-old/
    /opt/nfs/zimbra-backup-current/18_Zimbra_Backup_09-May-2008_DIFF.1.dar moved to /opt/nfs/zimbra-backup-old/
    /opt/nfs/zimbra-backup-current/18_Zimbra_Backup_09-May-2008_DIFF.1.dar.md5 moved to /opt/nfs/zimbra-backup-old/
    /opt/nfs/zimbra-backup-current/18_Zimbra_Backup_10-May-2008_DIFF.1.dar moved to /opt/nfs/zimbra-backup-old/
    /opt/nfs/zimbra-backup-current/18_Zimbra_Backup_10-May-2008_DIFF.1.dar.md5 moved to /opt/nfs/zimbra-backup-old/
    /opt/nfs/zimbra-backup-current/18_Zimbra_Backup_11-May-2008_DIFF.1.dar moved to /opt/nfs/zimbra-backup-old/
    /opt/nfs/zimbra-backup-current/18_Zimbra_Backup_11-May-2008_DIFF.1.dar.md5 moved to /opt/nfs/zimbra-backup-old/
    java.net.UnknownHostException: zim.chrishewitt.net: zim.chrishewitt.net
    at java.net.InetAddress.getLocalHost(InetAddress.java :1353)
    at com.zimbra.common.localconfig.LC.<clinit>(LC.java: 350)
    at com.zimbra.common.localconfig.KnownKey.<clinit>(Kn ownKey.java:32)
    at com.zimbra.common.localconfig.LocalConfig.expandAl l(LocalConfig.java:327)
    at com.zimbra.common.localconfig.LocalConfig.<init>(L ocalConfig.java:177)
    at com.zimbra.common.localconfig.LocalConfig.readConf ig(LocalConfig.java:343)
    at com.zimbra.common.localconfig.LocalConfig.<clinit> (LocalConfig.java:348)
    at com.zimbra.common.localconfig.Main.main(Main.java: 108)

    Setting date & version Marker into /opt/zimbra/DATE_VERSION.txt

    java.net.UnknownHostException: zim.chrishewitt.net: zim.chrishewitt.net
    at java.net.InetAddress.getLocalHost(InetAddress.java :1353)
    at com.zimbra.common.localconfig.LC.<clinit>(LC.java: 350)
    at com.zimbra.common.localconfig.KnownKey.<clinit>(Kn ownKey.java:32)
    at com.zimbra.common.localconfig.LocalConfig.expandAl l(LocalConfig.java:327)
    at com.zimbra.common.localconfig.LocalConfig.<init>(L ocalConfig.java:177)
    at com.zimbra.common.localconfig.LocalConfig.readConf ig(LocalConfig.java:343)
    at com.zimbra.common.localconfig.LocalConfig.<clinit> (LocalConfig.java:348)
    at com.zimbra.common.localconfig.Main.main(Main.java: 108)
    Doing a hotsync of /opt/zimbra/ to /tmp/zimbra-backup-tmp/
    sync went ok

    Stopping the Zimbra server

    Deactivated zimbra's cronjobs

    java.net.UnknownHostException: zim.chrishewitt.net: zim.chrishewitt.net
    at java.net.InetAddress.getLocalHost(InetAddress.java :1353)
    at com.zimbra.common.localconfig.LC.<clinit>(LC.java: 350)
    at com.zimbra.common.localconfig.KnownKey.<clinit>(Kn ownKey.java:32)
    at com.zimbra.common.localconfig.LocalConfig.expandAl l(LocalConfig.java:327)
    at com.zimbra.common.localconfig.LocalConfig.<init>(L ocalConfig.java:177)
    at com.zimbra.common.localconfig.LocalConfig.readConf ig(LocalConfig.java:343)
    at com.zimbra.common.localconfig.LocalConfig.<clinit> (LocalConfig.java:348)
    at com.zimbra.common.localconfig.Main.main(Main.java: 108)
    Host zim.chrishewitt.net
    Stopping stats...Done
    Stopping mta...Done
    Stopping spell...Done
    Stopping snmp...Done
    Stopping archiving...Done
    Stopping antivirus...Done
    Stopping antispam...Done
    Stopping imapproxy...Done
    Stopping mailbox...Done
    Stopping logger...Done
    Stopping ldap...Done

    Zimbra server has stopped

    kill any left over zimbra procs
    Did a Mr. Bush on some left over procs...

    Doing a fast cold sync...
    sync went ok

    Reinstated zimbra's cronjobs

    java.net.UnknownHostException: zim.chrishewitt.net: zim.chrishewitt.net
    at java.net.InetAddress.getLocalHost(InetAddress.java :1353)
    at com.zimbra.common.localconfig.LC.<clinit>(LC.java: 350)
    at com.zimbra.common.localconfig.KnownKey.<clinit>(Kn ownKey.java:32)
    at com.zimbra.common.localconfig.LocalConfig.expandAl l(LocalConfig.java:327)
    at com.zimbra.common.localconfig.LocalConfig.<init>(L ocalConfig.java:177)
    at com.zimbra.common.localconfig.LocalConfig.readConf ig(LocalConfig.java:343)
    at com.zimbra.common.localconfig.LocalConfig.<clinit> (LocalConfig.java:348)
    at com.zimbra.common.localconfig.Main.main(Main.java: 108)
    Host zim.chrishewitt.net
    Starting ldap...Done.
    Starting logger...Done.
    Starting mailbox...Done.
    Starting snmp...Done.
    Starting spell...Done.
    Starting mta...Done.
    Starting stats...Done.

    Zimbra has started

    Service down time was - Hr:0 Min:10 Sec:34

    Running A hack... This one to check and start Stats subsystem
    Hack not needed Stats seems to be running fine...

    All services are running


    Writing a full backup called:
    19_Zimbra_Backup_12-May-2008_FULL
    in: /opt/nfs/zimbra-backup-current/ with file sizes of max: 4395M
    Archive is NOT Encrypted!!!


    --------------------------------------------
    43236 inode(s) saved
    with 1124 hard link(s) recorded
    0 inode(s) changed at the moment of the backup
    0 inode(s) not saved (no inode/file change)
    0 inode(s) failed to save (filesystem error)
    0 inode(s) ignored (excluded by filters)
    0 inode(s) recorded as deleted from reference backup
    --------------------------------------------
    Total number of inode considered: 43236
    --------------------------------------------
    EA saved for 0 inode(s)
    --------------------------------------------
    Archive Created successfully

    Create MD5 Checksum for 19_Zimbra_Backup_12-May-2008_FULL.1.dar

    MD5 Checksum Created successfully


    Create file listing from archive

    :::::::::::::::::::::::::::::::::::::::::::::::::: :::::::::::::::::::
    Full Zimbra Backup ended at: 03:08
    Backup took Hr:1 Min:98 Sec:31 to complete
    :::::::::::::::::Cheers Osoffice for the script:::::::::::::::::::::::

    >>>>>>>>>>>>> END <<<<<<<<<<<<

    Note the strange elapsed time (as normal)..

    Any ideas?

    Christian

  6. #66
    heinzg is offline Loyal Member
    Join Date
    Jan 2008
    Location
    Germany
    Posts
    83
    Rep Power
    7

    Default

    Quote Originally Posted by chrishewitt View Post
    Hmmmm... I had the first "end of week" event overnight and things rolled over and I got the following email this morning. Any ideas on why the Java exception is thrown?

    >>>>>>>>>>> START <<<<<<<<<<<<<

    Full Backup started at: 01:30

    Old backups found...old week= 18 current week= 19
    ......
    java.net.UnknownHostException: zim.chrishewitt.net: zim.chrishewitt.net
    at java.net.InetAddress.getLocalHost(InetAddress.java :1353)
    at com.zimbra.common.localconfig.LC.<clinit>(LC.java: 350)
    at com.zimbra.common.localconfig.KnownKey.<clinit>(Kn ownKey.java:32)
    at com.zimbra.common.localconfig.LocalConfig.expandAl l(LocalConfig.java:327)
    at com.zimbra.common.localconfig.LocalConfig.<init>(L ocalConfig.java:177)
    at com.zimbra.common.localconfig.LocalConfig.readConf ig(LocalConfig.java:343)
    at com.zimbra.common.localconfig.LocalConfig.<clinit> (LocalConfig.java:348)
    at com.zimbra.common.localconfig.Main.main(Main.java: 108)

    Setting date & version Marker into /opt/zimbra/DATE_VERSION.txt

    java.net.UnknownHostException: zim.chrishewitt.net: zim.chrishewitt.net
    at java.net.InetAddress.getLocalHost(InetAddress.java :1353)
    at com.zimbra.common.localconfig.LC.<clinit>(LC.java: 350)
    at com.zimbra.common.localconfig.KnownKey.<clinit>(Kn ownKey.java:32)
    at com.zimbra.common.localconfig.LocalConfig.expandAl l(LocalConfig.java:327)
    at com.zimbra.common.localconfig.LocalConfig.<init>(L ocalConfig.java:177)
    at com.zimbra.common.localconfig.LocalConfig.readConf ig(LocalConfig.java:343)
    at com.zimbra.common.localconfig.LocalConfig.<clinit> (LocalConfig.java:348)
    at com.zimbra.common.localconfig.Main.main(Main.java: 108)

    ......


    :::::::::::::::::::::::::::::::::::::::::::::::::: :::::::::::::::::::
    Full Zimbra Backup ended at: 03:08
    Backup took Hr:1 Min:98 Sec:31 to complete
    :::::::::::::::::Cheers Osoffice for the script:::::::::::::::::::::::

    >>>>>>>>>>>>> END <<<<<<<<<<<<

    Note the strange elapsed time (as normal)..

    Any ideas?

    Christian
    Hi Christian!

    I have no idea why the Java exception is thrown. But it seems to happen when one calls 'zmcontrol' commands.
    Try running a "zmcontrol status" as the zimbra user. I bet you will get the same exceptions.

    About the funny time issue, I will have a look at that soon. I see the time stats a as a nice extra but not of real importance...( thus not that high on the list to fix in the limited time I currently have, sorry)

    heinzg
    Motions of a day at work -

  7. #67
    heinzg is offline Loyal Member
    Join Date
    Jan 2008
    Location
    Germany
    Posts
    83
    Rep Power
    7

    Default

    Quote Originally Posted by chrishewitt View Post
    # cat Backuplist-24-April-2008.txt
    FATAL error, aborting operation
    too large elastic buffer or elastic buffer incoherent structure

    Any ideas?

    Christian
    Hi Christian,

    have close look at the following two commands they show K & J options with crypt FULL & DIFF files and if the options are used on the wrong archive you will get the error above.

    Code:
    dar -K bf:MYLONGCRYPTKEY -l 19_Zimbra_Backup_13-May-2008_FULL  | head
    Warning, the archive 19_Zimbra_Backup_13-May-2008_FULL has been encrypted. A wrong key is not possible to detect, it would cause DAR to report the archive as corrupted
    [data ][ EA  ][compr] | permission | user  | group | size  |          date                 |    filename
    ----------------------+------------+-------+-------+-------+-------------------------------+------------
    [Saved]       [-----]   drwxr-xr-x   1001	zimbra	0	Tue May 13 20:16:30 2008	db
    [Saved]       [-----]   drwxr-x---   zimbra	zimbra	0	Tue May 13 19:57:26 2008	db/data
    [Saved]       [-----]   drwx------   zimbra	zimbra	0	Sun Mar 23 16:39:41 2008	db/data/test
    [Saved]       [-----]   drwx------   zimbra	zimbra	0	Sun Mar 23 16:39:42 2008	db/data/mysql
    [Saved]       [     ]   -rw-rw----   zimbra	zimbra	0	Sun Mar 23 16:39:42 2008	db/data/mysql/procs_priv.MYD
    [Saved]       [  82%]   -rw-rw----   zimbra	zimbra	1024	Sun Mar 23 16:39:42 2008	db/data/mysql/procs_priv.MYI
    [Saved]       [  94%]   -rw-rw----   zimbra	zimbra	8875	Sun Mar 23 16:39:42 2008	db/data/mysql/procs_priv.frm
    and

    Code:
    dar -J bf:MYLONGCRYPTKEY -l 19_Zimbra_Backup_13-May-2008_DIFF | head
    -J is only useful with -A option, for the archive of reference
    [data ][ EA  ][compr] | permission | user  | group | size  |          date                 |    filename
    ----------------------+------------+-------+-------+-------+-------------------------------+------------
    [Saved]       [-----]   drwxr-xr-x   1001	zimbra	0	Tue May 13 21:40:48 2008	db
    [Saved]       [-----]   drwxr-x---   zimbra	zimbra	0	Tue May 13 20:18:22 2008	db/data
    [     ]       [-----]   drwx------   zimbra	zimbra	0	Sun Mar 23 16:39:41 2008	db/data/test
    [     ]       [-----]   drwx------   zimbra	zimbra	0	Sun Mar 23 16:39:42 2008	db/data/mysql
    [     ]       [-----]   -rw-rw----   zimbra	zimbra	0	Sun Mar 23 16:39:42 2008	db/data/mysql/procs_priv.MYD
    [     ]       [-----]   -rw-rw----   zimbra	zimbra	1024	Sun Mar 23 16:39:42 2008	db/data/mysql/procs_priv.MYI
    [     ]       [-----]   -rw-rw----   zimbra	zimbra	8875	Sun Mar 23 16:39:42 2008	db/data/mysql/procs_priv.frm
    [     ]       [-----]   -rw-rw----   zimbra	zimbra	77815	Sun Mar 23 16:39:42 2008	db/data/mysql/help_keyword.MYD
    Now the error:
    Code:
    dar -K bf:MYLONGCRYPTKEY -l 19_Zimbra_Backup_13-May-2008_DIFF | head
    FATAL error, aborting operation
    too large elastic buffer or elastic buffer incoherent structure
    Does this help you, Please check this agains your dar files, and let me know. That is if you still have some dar that are crypt.

    heinzg
    Motions of a day at work -

  8. #68
    gilus is offline Member
    Join Date
    Aug 2007
    Posts
    10
    Rep Power
    7

    Default Rsync question,

    Hi,

    First of all: GREAT TOOL! Thanks for sharing!

    Unfortunately I' finding in the log files lines like this:
    file has vanished: "/opt/zimbra/store/0/30/msg/8/34938-143841.msg"

    Is it rsync error?
    (Ububntu 6.06 LTS, Zimbra 5)

    Thanks for any ideas,
    Michal

  9. #69
    heinzg is offline Loyal Member
    Join Date
    Jan 2008
    Location
    Germany
    Posts
    83
    Rep Power
    7

    Default

    Quote Originally Posted by gilus View Post
    Hi,

    First of all: GREAT TOOL! Thanks for sharing!

    Unfortunately I' finding in the log files lines like this:
    file has vanished: "/opt/zimbra/store/0/30/msg/8/34938-143841.msg"

    Is it rsync error?
    (Ububntu 6.06 LTS, Zimbra 5)

    Thanks for any ideas,
    Michal
    Hi Michal!

    Cheers for the thanx!

    Your question: Nope it is not to worry about.
    This happens during hot syncs. in your case the email "34938-143841.msg" was deleted during hot sync. How it works is when rsync starts it builds an index of all the files it is going to transfer, and after that another of the transfered file which should match. But due to the file being delete between indexes it must of vanished .

    But you should never have this during cold sync i.e. when the server is stopped to sync again.

    Hope that helps.

    Heinzg
    Motions of a day at work -

  10. #70
    gilus is offline Member
    Join Date
    Aug 2007
    Posts
    10
    Rep Power
    7

    Default Thanks!!!

    Hi Heinzg,

    Thanks again for fast reply! Indeed it happens only while hotsync (I guess it was because some of the users were still online and working! ;-) Cold sync went without any errors.

    Again than You for a great tool! The people like You are the core of the Open Source idea! Thanks for the support and good luck!

    Michal

Page 7 of 66 FirstFirst ... 567891757 ... LastLast

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. Zimbra Backup Script ... Small Problem
    By frankb in forum Administrators
    Replies: 12
    Last Post: 07-14-2008, 08:40 AM
  2. Debian - Upgrade from 4.5.6
    By magikman in forum Installation
    Replies: 1
    Last Post: 01-19-2008, 12:59 PM
  3. Backup script issues
    By SSS in forum Administrators
    Replies: 18
    Last Post: 12-03-2007, 09:56 AM
  4. 4.0.4 to 4.5.6 upgrade failed in network edition
    By chenthil in forum Administrators
    Replies: 1
    Last Post: 08-27-2007, 09:36 AM
  5. [SOLVED] Simple backup question...
    By dameron in forum Administrators
    Replies: 3
    Last Post: 08-25-2007, 09:36 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •