Results 1 to 7 of 7

Thread: --zipStore doesn't work.

  1. #1
    lcxe is offline Active Member
    Join Date
    May 2008
    Posts
    34
    Rep Power
    7

    Default --zipStore doesn't work.

    We've been trying to get our backups into a state where we would be able to easily rsync them from our live server over to our backup server. The problem is that to date we've been unable to move them over in a reasonable amount of time due to the number of files.

    Somewhat recently we found out about the --zip and --zipStore switches for the zmbackup command. From what we read everything indicated that the zip options would put all the blobs into zip files, drastically reducing the number of files in the backups, because the blobs would be clumped into the zip files.

    However after running the backup command with the --zip option was out of the question for us as when we tried using it, by the time we go into the office at nine in the morning the full backup had no completed after eight hours. In comparison our normal full backups typically take around half an hour to complete. We then attempted to run the backup command with the --zipStore option. This was much more successful, as it took almost the time time as the normal full backup took run. But after comparing the number of files contained within the two full backups, there was almost no difference.

    Here is the output for the first unzipped backup file:
    Code:
    zimbra@server:~/backup/sessions$ nice ls -R full-20081202.060006.336 | wc
    1454668 1370892 60737837
    And the output for the zipStore backup file:
    Code:
    zimbra@server:~/backup/sessions$ nice ls -R full-20081203.163307.186 | wc
    1461372 1376489 60965519
    Further more when I explored into the zipStore backup file, there was no sign of any zips.

    For the record the commands used to create the backups were:
    Code:
    zimbra@pat01:~$ zmbackup -f -a all
    zimbra@pat01:~$ zmbackup -f --zipStore -a all

  2. #2
    Jesster's Avatar
    Jesster is offline Trained Alumni
    Join Date
    Nov 2007
    Location
    Santa Barbara, CA.
    Posts
    109
    Rep Power
    7

    Default

    Hi lcxe,

    I feel your pain! I had similar issues, until I discovered it's the combination of _both_ arguments to get few files and faster backups.


    use --zip --zipStore together!

    HTH
    -Jessie

  3. #3
    Dirk's Avatar
    Dirk is offline Moderator
    Join Date
    May 2006
    Location
    England.
    Posts
    927
    Rep Power
    10

    Default

    The help for zmbackup shows this:
    Code:
    -z,--zip                 Backs up blobs in zip files rather than as
                              individual files.
        --zipStore            When --zip option is used, store blobs
                              uncompressed.
    my crontab has the command listed with just -z which the help would seem to indicate to be synonimous with both --zip and --zipStore, but the text of the help implies that --zip is an uncompressed archive and --zipStore is compressed.

    I'd say that's the wrong way around anyway, --zipStore should be the uncompressed one (store the files, dont compress them)

    So I'm still confused.

  4. #4
    lcxe is offline Active Member
    Join Date
    May 2008
    Posts
    34
    Rep Power
    7

    Default

    Thank you very much for your reply Jesster, that is very helpful knowledge. I doubt I would've figured that one out no matter what I did.

    I guess it would help if the command threw some sort of error message if you didn't use the --zip option along with --zipStore, because letting it proceed as if it's working it just confusing.

    And to Dirk, zipStore is the switch that leaves the files uncompressed in zip format. Just plain zip will compress the files and put them into a zip format. I think the phrasing is a little poor for the description of zipStore. Reading it now I see that it tells you what Jesster had said about using the two in conjunction, and what it does. But alone it appears as something completely different.

  5. #5
    Jesster's Avatar
    Jesster is offline Trained Alumni
    Join Date
    Nov 2007
    Location
    Santa Barbara, CA.
    Posts
    109
    Rep Power
    7

    Default

    Quote Originally Posted by lcxe View Post
    Thank you very much for your reply Jesster, that is very helpful knowledge. I doubt I would've figured that one out no matter what I did.
    Glad to have helped! We struggled on the very same problem. I'm surprised the default backup options don't use --zip --zipStore because it's more efficient on the filesystem and archives.

  6. #6
    lcxe is offline Active Member
    Join Date
    May 2008
    Posts
    34
    Rep Power
    7

    Default

    Using zipStore seems to be acting just like it did with the zip option now. I was hopeful when you mentioned it being fast in your description Jesster, but it doesn't seem to be the case for us.

    We only ran it for an hour this morning to see how it compared, and in that hour it ran, it wasn't able to get by the first account it was to backup. Mind you the account is 44GB, but still. We were seeing all the accounts take a little more than 20 minutes on a normal full backup.

    I feel like I'm missing something though. Because currently our full backups are only 106GB, and I'm seeing people talking about having systems with tens of thousands of users, compared to our ~40. How are they able to get their backups done in a reasonable amount of time so that their users on their system don't suffer from downtime every time a backup is done? I know they probably have multi-server systems in place to deal with this, but the underlying idea is there. How do others get their backups done with single system setups, with a decently large amount of data in a feasible amount of time?

  7. #7
    Jesster's Avatar
    Jesster is offline Trained Alumni
    Join Date
    Nov 2007
    Location
    Santa Barbara, CA.
    Posts
    109
    Rep Power
    7

    Default

    We switched to Auto-Grouped backups from Standard, this helped reduce the amount of users to backup on a daily basis and the backup window needed at night.

    There's a good write up here: Backup and Restore we have several thousands of users, so Auto-Grouped is kind of a necessity.

    Are you writing the backups to another disk (separate disk from /opt/zimbra/) ?

    do you have Zimbra stats package installed & enabled?

    if so you can create some charts to help see if there's any IO issues, I think the program is zmstat-chart, more about that here: Server Monitoring - Zimbra :: Wiki

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. Rules Outlook 2003 or 2007 not work with Zimbra
    By Ahmad Hafiz Fikry in forum Administrators
    Replies: 1
    Last Post: 11-20-2008, 12:52 AM
  2. Zimbra Whitelist techniques don't work
    By fireman949 in forum Administrators
    Replies: 4
    Last Post: 04-13-2008, 12:05 AM
  3. Replies: 8
    Last Post: 12-02-2007, 05:18 AM
  4. Replies: 42
    Last Post: 08-11-2006, 09:50 AM
  5. HELP- Making Zimbra work. after installation
    By hifriend1 in forum Installation
    Replies: 7
    Last Post: 04-05-2006, 06:43 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •