Results 1 to 8 of 8

Thread: Live Archive of Old Users - Attempt 2

  1. #1
    quietas is offline Elite Member
    Join Date
    Aug 2007
    Location
    Anchorage, AK
    Posts
    376
    Rep Power
    7

    Default Live Archive of Old Users - Attempt 2

    I've asked this before and I was never able to get good specifics. I'll layout my hardware and software first:

    Mail:
    Outside IP behind DMZ
    Domain, single mail server
    Ubuntu 6.06
    Zimbra NE 5.0.11

    Test:
    Outside IP behind DMZ
    Subdomain, single mail server
    Ubuntu 8.04
    Zimbra FOSS 5.0.11

    Essentially I need to find a way to move individual user accounts from the NE server to the FOSS server.

    I have the backups from the NE box, but of course I can not us that backup/restore on the FOSS unit. We want the FOSS so that we can have the unlimited users where our NE box fills up due to employee turnover.

    I need to find a way to move a user account, mail, calendar, contacts, the whole nine yards. With high turnover my managers want to keep a record of what was done by whom and have notes for future employees to look at if needed.
    ---------------------
    ---------------------

    I'm updating here with what I need to do with the current script posted in the code box below. The posts below are problems or solutions and comments as I am working on this.

    Still to do:
    • DONE: Script via SSH the restore remotely (possibly on hourly basis via cron?)
    • DONE: Mark restored accounts as closed or locked on archive unit via script
    • DONE: Currently script needs to run on production unit as Zimbra user, make possible via sudo and root maybe?
    • .
    • Create a way to transfer all account info: Name, phone number, address, ....
    • Add test and verification that user was restored on archive unit
    • Create Zimlet for easy use by my non-CLI friendly admins (ie. my boss)


    Use "sudo su zimbra - x -c /opt/zimbra/bin/zmuserback username" to run as the Zimbra user account. Easy to script this for each admin user for a simple command like ./zmarchive or something similar.

    Code:
    #!/bin/bash
    # zmuserback -- Script to export select Zimbra accounts
    # The above is called using "zmuserback username"
    # A file username@domain.com.tgz is created in the directory specified.
    # It then will use SCP to copy the file to the new server.
    # Using SSH commands will run to create and restore the account
    
    USERNAME=$1
    DOMAIN=domain.com
    NEWDOMAIN=test.domain.com
    NEWSERVER=123.123.123.123          #I used IP address rather than DNS
    TODAY=`date`
    LOGFILE="/opt/zimbra/backup/zmback/zmbackuplog.txt"
    
    if [ -z $USERNAME ]; then
        echo Please enter a username next attempt
        exit
    fi
    
    echo --------------------- >> $LOGFILE
    echo Deleting previous backup file for $USERNAME@$DOMAIN
    rm -f /opt/zimbra/backup/zmback/$USERNAME@$DOMAIN.tgz
    
    echo Exporting $USERNAME - $TODAY
    echo Exporting $USERNAME - $TODAY >> $LOGFILE
    /opt/zimbra/bin/zmmailbox -z -m $USERNAME@$DOMAIN gms >> $LOGFILE
    /opt/zimbra/bin/zmmailbox -z -m $USERNAME@$DOMAIN getRestURL "//?fmt=tgz" > /opt/zimbra/backup/zmback/$USERNAME@$DOMAIN.tgz
    scp /opt/zimbra/backup/zmback/$USERNAME@$DOMAIN.tgz zimbra@$NEWSERVER:/opt/zimbra/backup/
    
    #Commented out the delete function while testing
    #echo Delete $USERNAME@$DOMAIN
    #/opt/zimbra/bin/zmprov deleteAccount $USERNAME@$DOMAIN >> $LOGFILE
    
    echo Creating user account for $USERNAME on $NEWDOMAIN
    ssh zimbra@$NEWSERVER /opt/zimbra/bin/zmprov deleteAccount $USERNAME@$NEWDOMAIN >> $LOGFILE
    ssh zimbra@$NEWSERVER /opt/zimbra/bin/zmprov createAccount $USERNAME@$NEWDOMAIN 'p@sSw0rd1234' >> $LOGFILE
    
    echo Restoring user account fot $USERNAME on $NEWDOMAIN
    ssh zimbra@$NEWSERVER /opt/zimbra/bin/zmmailbox -z -m "$USERNAME@$NEWDOMAIN postRestURL '//?fmt=tgz&resolve=reset' /opt/zimbra/backup/$USERNAME@$DOMAIN.tgz"
    ssh zimbra@$NEWSERVER rm /opt/zimbra/backup/$USERNAME@$DOMAIN.tgz
    
    echo Locking $USERNAME@$NEWDOMAIN
    ssh zimbra@$NEWSERVER /opt/zimbra/bin/zmprov modifyAccount $USERNAME@$NEWDOMAIN zimbraAccountStatus closed
    
    echo --------------------- >> $LOGFILE
    This is the basic script I set up to run the main script as the Zimbra user.
    Code:
    #!/bin/bash
    # zmarchive
    # This basic script runs the zmuserback script as the Zimbra user.
    # It will compress the specified user.
    # The compressed file is sent to the test server for a live archive.
    # The account will be deleted from the production server.
    # Commands will be issued via SSH to restore the account on the archive server.
    # The Archive account is then closed.
    
    USER=$1
    
    if [ -z $USER ]; then
        echo Please enter a username next attempt
        exit
    fi
    
    
    sudo su zimbra -c /opt/zimbra/bin/zmuserback $USER
    Last edited by quietas; 01-13-2009 at 04:08 PM.
    Culley
    Mail | Dell 2950III | 2x Quad Core 5420 | 8gb RAM | 6x 146gb SAS RAID 0+1 | Red Hat 5.3 | Zimbra 6.0.10 Network Edition
    Test | VMware ESXi Whitebox | Phenom II Black 3.2ghz | 12gb RAM | 6x 1tb SATA RAID 0+1 | CentOS 5.4 | FOSS, Not in use now

  2. #2
    quietas is offline Elite Member
    Join Date
    Aug 2007
    Location
    Anchorage, AK
    Posts
    376
    Rep Power
    7

    Default

    I set up a samba share between the two servers (DMZ IP to IP only via firewall), and tried the command at: PowerTips - Admins » Zimbra :: Blog

    sudo /opt/zimbra/bin/zmmailbox -z -m atest@domain.com getRestURL .//?fmt=tgz. > /opt/zimbra/samba-archive/atest20080112.tgz
    ERROR: service.FAILURE (system failure: GET failed, status=501 not implemented yet)
    I can't seem to find documentation on this error.

    Edit: I copied and pasted the command from Firefox to Putty and it seems that it changed "//?fmt=tgz" to .//?fmt=tgz. for some reason. I put the quotes in and that part works. Now how can I automate this. =)
    Last edited by quietas; 01-12-2009 at 02:10 PM.
    Culley
    Mail | Dell 2950III | 2x Quad Core 5420 | 8gb RAM | 6x 146gb SAS RAID 0+1 | Red Hat 5.3 | Zimbra 6.0.10 Network Edition
    Test | VMware ESXi Whitebox | Phenom II Black 3.2ghz | 12gb RAM | 6x 1tb SATA RAID 0+1 | CentOS 5.4 | FOSS, Not in use now

  3. #3
    uxbod's Avatar
    uxbod is offline Moderator
    Join Date
    Nov 2006
    Location
    UK
    Posts
    8,017
    Rep Power
    24

    Default

    Run it from cron ? If you put it in a script and then once the zmmailbox command has completed just create a flag file, and then on the remote server have another cron job that checks for that flag file and once it finds it then run the inverse command to import the data.

  4. #4
    quietas is offline Elite Member
    Join Date
    Aug 2007
    Location
    Anchorage, AK
    Posts
    376
    Rep Power
    7

    Default

    With a bit of work and mooching of other people basic scripts I cam up with a way to archive from an NE box over to a FOSS system with out using any NE tools that the FOSS unit does not have.

    First you need to be able to SSH or SCP without entering a password to be able to script the backup transfer without intervention. Follow the steps here: HowTo SSH/SCP without a password.

    I used SGIA's basic script for copying a user entirely and modified it a bit: http://www.zimbra.com/forums/adminis...tml#post110804

    It looks like this needs Zimbra 5.0.10 or higher, as 5.0.9 and below may be susceptible to minor corruption in the tar/gzip process.

    Code:
    #!/bin/bash
    # zmuserback -- Script to export select Zimbra accounts
    # The above is called using "zmuserback username"
    # and outputs a file username@domain.tld.tgz to the directory specified.
    
    USERNAME=$1
    DOMAIN=@domain.tld
    TODAY=`date`
    LOGFILE="/opt/zimbra/backup/zmback/zmbackuplog.txt"
    
    echo --------------------- >> $LOGFILE
    echo Export $USERNAME - $TODAY >> $LOGFILE
    /opt/zimbra/bin/zmmailbox -z -m $USERNAME$DOMAIN gms >> $LOGFILE
    /opt/zimbra/bin/zmmailbox -z -m $USERNAME$DOMAIN getRestURL "//?fmt=tgz" > /opt/zimbra/backup/zmback/$USERNAME$DOMAIN.tgz
    scp /opt/zimbra/backup/zmback/$USERNAME$DOMAIN.tgz zimbra@123.123.123.123:/opt/zimbra/backup/
    rm -f /opt/zimbra/backup/zmback/$USERNAME$DOMAIN.tgz
    echo --------------------- >> $LOGFILE
    And on the receiving end:
    Code:
    #!/bin/bash
    # zmuserrestore -- Script to restore select Zimbra accounts
    # The above is called using "zmuserrestore username"
    
    USERNAME=$1
    OLDDOMAIN=@domain.tld
    NEWDOMAIN=@archive.domain.tld
    TODAY=`date`
    LOGFILE="/opt/zimbra/backup/backuplog.txt"
    
    echo --------------------- >> $LOGFILE
    echo Creating user account for $USERNAME
    zmprov deleteAccount $USERNAME$NEWDOMAIN >> $LOGFILE
    zmprov createAccount $USERNAME$NEWDOMAIN 'p@sSw0rd1234' >> $LOGFILE
    echo Restore $USERNAME >> $LOGFILE
    /opt/zimbra/bin/zmmailbox -z -m $USERNAME$NEWDOMAIN postRestURL "//?fmt=tgz&resolve=reset" /opt/zimbra/backup/$USERNAME$OLDDOMAIN.tgz
    rm /opt/zimbra/backup/$USERNAME$OLDDOMAIN.tgz
    echo --------------------- >> $LOGFILE
    echo Done creating $USERNAME$NEWDOMAIN
    It simply copies and archives the user specified to a tgz file and then copies it via SCP to the target archive system. Then on the archive system you run the second script which then restores it, leaving you with a live archive copy.

    Still to do:
    • DONE: Script via SSH the restore remotely (possibly on hourly basis via cron?)
    • Currently script needs to run on production unit as Zimbra user, make possible via sudo and root maybe?
    • Add test and verification that user was restored on archive unit
    • Mark restored accounts as closed or locked on archive unit via script
    • Create Zimlet for easy use by my non-CLI friendly admins (ie. my boss)
    Last edited by quietas; 01-13-2009 at 11:20 AM.
    Culley
    Mail | Dell 2950III | 2x Quad Core 5420 | 8gb RAM | 6x 146gb SAS RAID 0+1 | Red Hat 5.3 | Zimbra 6.0.10 Network Edition
    Test | VMware ESXi Whitebox | Phenom II Black 3.2ghz | 12gb RAM | 6x 1tb SATA RAID 0+1 | CentOS 5.4 | FOSS, Not in use now

  5. #5
    quietas is offline Elite Member
    Join Date
    Aug 2007
    Location
    Anchorage, AK
    Posts
    376
    Rep Power
    7

    Default

    It looks like I can run most of the remote script with SSH. One part is not working atm though.

    ssh zimbra@123.123.123.123 /opt/zimbra/bin/zmmailbox -z -m username@archive.domain.tld postRestURL "//?fmt=tgz&resolve=reset" /opt/zimbra/backup/username@domain.tld.tgz
    bash: /opt/zimbra/backup/username@domain.tld.tgz: Permission denied
    I ran it manually without using the variables and I got a Permission denied error, but the same command run locally as the Zimbra user works while it does not using SSH.

    EDIT: Figured it out, changed to:
    ssh zimbra@123.123.123.123 /opt/zimbra/bin/zmmailbox -z -m "username@archive.domain.tld[/email] postRestURL '//?fmt=tgz&resolve=reset' /opt/zimbra/backup/username@domain.tld.tgz"
    It was trying to execute it as multiple commands after the "s
    Last edited by quietas; 01-13-2009 at 10:20 AM.
    Culley
    Mail | Dell 2950III | 2x Quad Core 5420 | 8gb RAM | 6x 146gb SAS RAID 0+1 | Red Hat 5.3 | Zimbra 6.0.10 Network Edition
    Test | VMware ESXi Whitebox | Phenom II Black 3.2ghz | 12gb RAM | 6x 1tb SATA RAID 0+1 | CentOS 5.4 | FOSS, Not in use now

  6. #6
    quietas is offline Elite Member
    Join Date
    Aug 2007
    Location
    Anchorage, AK
    Posts
    376
    Rep Power
    7

    Default

    I seem to have it working via SSH now.

    Code:
    #!/bin/bash
    # zmuserback -- Script to export select Zimbra accounts
    # The above is called using "zmuserback username"
    # A file username@domain.com.tgz is created in the directory specified.
    # It then will use SCP to copy the file to the new server.
    # Using SSH commands will run to create and restore the account
    
    USERNAME=$1
    DOMAIN=domain.com
    NEWDOMAIN=test.domain.com
    NEWSERVER=123.123.123.123          #I used IP address rather than DNS
    TODAY=`date`
    LOGFILE="/opt/zimbra/backup/zmback/zmbackuplog.txt"
    
    echo --------------------- >> $LOGFILE
    echo Deleting previous backup file for $USERNAME@$DOMAIN
    rm -f /opt/zimbra/backup/zmback/$USERNAME@$DOMAIN.tgz
    
    echo Exporting $USERNAME - $TODAY
    echo Exporting $USERNAME - $TODAY >> $LOGFILE
    /opt/zimbra/bin/zmmailbox -z -m $USERNAME@$DOMAIN gms >> $LOGFILE
    /opt/zimbra/bin/zmmailbox -z -m $USERNAME@$DOMAIN getRestURL "//?fmt=tgz" > /opt/zimbra/backup/zmback/$USERNAME@$DOMAIN.tgz
    scp /opt/zimbra/backup/zmback/$USERNAME@$DOMAIN.tgz zimbra@NEWDOMAIN:/opt/zimbra/backup/
    
    echo Creating user account for $USERNAME on $NEWDOMAIN
    ssh zimbra@$NEWSERVER /opt/zimbra/bin/zmprov deleteAccount $USERNAME@$NEWDOMAIN >> $LOGFILE
    ssh zimbra@$NEWSERVER /opt/zimbra/bin/zmprov createAccount $USERNAME@$NEWDOMAIN 'p@sSw0rd1234' >> $LOGFILE
    
    echo Restoring user account fot $USERNAME on $NEWDOMAIN
    ssh zimbra@$NEWSERVER /opt/zimbra/bin/zmmailbox -z -m "$USERNAME@$NEWDOMAIN postRestURL '//?fmt=tgz&resolve=reset' /opt/zimbra/backup/$USERNAME@$DOMAIN.tgz"
    ssh zimbra@$NEWSERVER rm /opt/zimbra/backup/$USERNAME@$DOMAIN.tgz
    echo --------------------- >> $LOGFILE
    Last edited by quietas; 01-13-2009 at 11:19 AM.
    Culley
    Mail | Dell 2950III | 2x Quad Core 5420 | 8gb RAM | 6x 146gb SAS RAID 0+1 | Red Hat 5.3 | Zimbra 6.0.10 Network Edition
    Test | VMware ESXi Whitebox | Phenom II Black 3.2ghz | 12gb RAM | 6x 1tb SATA RAID 0+1 | CentOS 5.4 | FOSS, Not in use now

  7. #7
    quietas is offline Elite Member
    Join Date
    Aug 2007
    Location
    Anchorage, AK
    Posts
    376
    Rep Power
    7

    Default

    Found the command to close the account using zmprov. It was in the wiki documentation of all places. Crazy.

    ModifyAccount - ma
    {name@domain|id|adminName} [attribute1 value1 etc]
    zmprov ma joe@domain.com zimbraAccountStatus maintenance

    echo Locking $USERNAME@$NEWDOMAIN
    ssh zimbra@$NEWSERVER /opt/zimbra/bin/zmprov modifyAccount $USERNAME@$NEWDOMAIN zimbraAccountStatus locked
    Also found how to run the command under another account using the su command. I use sudo since I am on Ubuntu and have not changed the random password which it the Zimbra user account was assigned.

    sudo su zimbra - x -c /opt/zimbra/bin/zmuserback $USERNAME
    Last edited by quietas; 01-13-2009 at 12:10 PM.
    Culley
    Mail | Dell 2950III | 2x Quad Core 5420 | 8gb RAM | 6x 146gb SAS RAID 0+1 | Red Hat 5.3 | Zimbra 6.0.10 Network Edition
    Test | VMware ESXi Whitebox | Phenom II Black 3.2ghz | 12gb RAM | 6x 1tb SATA RAID 0+1 | CentOS 5.4 | FOSS, Not in use now

  8. #8
    quietas is offline Elite Member
    Join Date
    Aug 2007
    Location
    Anchorage, AK
    Posts
    376
    Rep Power
    7

    Default

    This is the basic script I set up to run the main script as the Zimbra user. I copied this to my /bin and run it with "zmarchive username".
    Code:
    #!/bin/bash
    # zmarchive
    # This basic script runs the zmuserback script as the Zimbra user.
    # It will compress the specified user.
    # The compressed file is sent to the test server for a live archive.
    # The account will be deleted from the production server.
    # Commands will be issued via SSH to restore the account on the archive server.
    # The Archive account is then closed.
    
    USER=$1
    
    sudo su zimbra -c /opt/zimbra/bin/zmuserback $USER
    Culley
    Mail | Dell 2950III | 2x Quad Core 5420 | 8gb RAM | 6x 146gb SAS RAID 0+1 | Red Hat 5.3 | Zimbra 6.0.10 Network Edition
    Test | VMware ESXi Whitebox | Phenom II Black 3.2ghz | 12gb RAM | 6x 1tb SATA RAID 0+1 | CentOS 5.4 | FOSS, Not in use now

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. Replies: 5
    Last Post: 09-27-2007, 02:27 PM
  2. need advice on configuring zimbra to work with fax server
    By pheonix1t in forum Administrators
    Replies: 0
    Last Post: 07-11-2007, 07:46 PM
  3. Archive mail for specific users
    By rrsd in forum Administrators
    Replies: 2
    Last Post: 09-25-2006, 12:50 PM
  4. Replies: 4
    Last Post: 01-13-2006, 07:54 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •