Results 1 to 4 of 4

Thread: [SOLVED] collapsing message volumes

  1. #1
    Vladimir is offline Advanced Member
    Join Date
    Aug 2007
    Posts
    220
    Rep Power
    7

    Default [SOLVED] collapsing message volumes

    I understand how one adds message volumes, but how do you ever remove them while maintaining the data on them? Lets say I start with one Xgb volume, add a second volume of Ygb and at a later point I want to replace both with a third volume of (X+Y+Z)gb.

    Is there a way to migrate messages from one to the other or do I just have to play games with directory names and linking both volumes to directories on the new partition?

  2. #2
    mmorse's Avatar
    mmorse is offline Moderator
    Join Date
    May 2006
    Location
    USA
    Posts
    6,242
    Rep Power
    21

    Default

    Solution1: Same Contents
    If you want new store2 to hold the contents of the old store in entirely as well - just move the blobs with zimbra stopped first.
    Then point the message1 path at /mount/store2. (Don't forget to check file permissions after the move.)
    It may be better to use zmmailboxmove so you don't cross user 'folders' ie on the filesystem your now sharing dirs Bug 18850 - multiple "current" primary, secondary, index zmvolumes

    Example: Volumes : Ajcody-Notes - Zimbra :: Wiki


    Solution2: Old as subdirectory of new

    (Having paths for both volume 1 & 2 point to disk 3 is common - better if you make it a subdirectory.)

    If you've already created store2 and been using both for a while, just make the old store1 a subdirectory of the new store2 @ on the new storage device:
    /mount/newstore2
    /mount/store/oldstore1 (previously /opt/zimbra/store - change the path in zmvolume)




    Solution3: Another method that can be used is updating the DB itself.

    There's a volume_id column in the mail_item table in every mboxgroup database - after moving the blobs themselves you could run something like: mysql> for (N = 1 through 100) {UPDATE mboxgroupN.mail_item SET volume_id = IDofNew WHERE volume_id = IDofOld;}

    The first N is talking about the mailbox group (id divided by 100) for which there's 100max groups.

    Have a read of: Account mailbox database structure - Zimbra :: Wiki

    If you wanted to poke around, some basic structure:
    su - zimbra
    mysql
    show databases;
    use mboxgroupN
    show tables;
    select * from mail_item where volume_id=1; (would get you all the items in volume 1)
    That's a BIG query btw! So probably better if you selected by mailbox_id or id - narrow this down and try stuff on a test account or individual msg - better yet a test harness machine before you run any update/set/alter commands across every account on your production system.

    CLI zmvolume - Zimbra :: Wiki
    zmvolume -l
    zmvolume -dc
    mysql> select * from zimbra.volume;

    MySQL commands: Pantz.org Technical Reference Site


    Solution4: Use HSM to move the blobs after x time from store1 > store2.

    ---

    re you saying LVM to size volume groups/pools by absorbing new physical disks/volumes to present to the system as one instance?

    As far as Zimbra is concerned:
    -Using LVM to expand /opt/zimbra, /opt/zimbra/store, or /opt/zimbra/index you don't have to do anything else.

    -Haven't implemented LVM etc? Don't worry, you can still create 'expanding mail storage' even if not presented to the system as one big unit. If your creating an /opt/zimbra/store2, /mnt/device/zimbra/store2, or /mnt/device/zimbra/index2 that's separate - in order to get Zimbra to start putting blobs there you need to point the current message/index or volume there.

    Just realize blobs are NOT moved in this process, you're only pointing at the location to store any NEW data. (See below explanation.)

    Thus you shouldn't remove the old store, but alternatively if you want new store2 to hold the contents of the old store as well - just move the blobs first as in 1A.

    Explanation:

    Each Zimbra mailbox server is configured with one current index & message (NE can add secondary HSM volumes).

    When a new message is delivered or created, the message is saved in the current message volume. Additional message volumes can be created, but only one is configured as the current volume where new messages are stored. When the volume is full, you can configure a new current message volume. The current message volume receives all new messages. New messages are never stored in the previous volume. A current volume cannot be deleted. If there are accounts with messages referencing a volume it should not be deleted until you point the volume path to wherever you've now moved the items.

    Each mailbox is assigned to a permanent index directory on the current index volume. When an account is created, the current index volume is automatically defined for the account. You cannot change which index volume the account is assigned. As volumes become full, you can create a new current index volume for new accounts. When a new index volume is added as current, the older index volume is no longer assigned new accounts. Index volumes not marked current are still actively in use as the index volumes for accounts assigned to them. Any index volume that is referenced by a mailbox as it's index volume cannot be deleted.

    The Network Edition has HSM (Hierarchical Storage Management) which involves moving data to other storage locations automatically after x amount of time. Messages and attachments are moved from a primary volume to the current secondary volume based on the age of the message; completely transparent to the user.

    For instance: After 30 days messages move from your fast (and usually more expensive) SCSI disks, to a SATA array so that you can keep storage costs down. But you could also use it to move between your 64GB & 320GB volumes if they're separate.

    Bug 18850 - multiple "current" primary, secondary, index zmvolumes
    Bug 18720 - Add support for more than one current secondary storage volume in HSM
    Bug 32719 - Add ability for admins to configure what blobs get migrated with HSM

    Bug 23472 - Ability to move/consolidate messages from one message store to another.
    Bug 35142 - volume migration tool

    If your merging two systems it may be better to use zmmailboxmove so you don't cross user 'folders' ie on the filesystem will go into the 'current'/ you don't want to share user level dirs. To solve this we really should append a zimbraId string to each users blob folder path (after the 100 group number), or even make it entirely like we do for backups.
    Last edited by mmorse; 04-28-2010 at 01:47 PM.

  3. #3
    Klug's Avatar
    Klug is offline Moderator
    Join Date
    Mar 2006
    Location
    Beaucaire, France
    Posts
    2,316
    Rep Power
    13

    Default

    AFAIK, you can also :
    . stop ZCS
    . move the messages by hand to a new filesystem/volume
    . start ZCS (in a way it does not receive mails and users can not connect to it)
    . change the path of the volume in the webadmin or CLI (you keep the same name but change the path)
    . restart ZCS (in a normal way)

    This method allows you not to do changes in the database.

  4. #4
    mayk.backus is offline Intermediate Member
    Join Date
    Apr 2008
    Location
    the Netherlands
    Posts
    22
    Rep Power
    7

    Default

    Quote Originally Posted by Klug View Post
    AFAIK, you can also :
    . stop ZCS
    . move the messages by hand to a new filesystem/volume
    . start ZCS (in a way it does not receive mails and users can not connect to it)
    . change the path of the volume in the webadmin or CLI (you keep the same name but change the path)
    . restart ZCS (in a normal way)

    This method allows you not to do changes in the database.
    Hi Klug,

    thank you for this interesting pointer.. Is this method tested ?
    Can i move the complete /store folder to a new location or must i create a new one and move it ?
    i presume changing the path of the volume is done with zmvolume -e ?

    I'm asking this because i'm about te have about 100 extra users on the server, and want some extra space on some external storage.

    When i create a new volume , the new volume will be the current volume and so receive all the new incoming mail.

    Could it be possible to define which volume holds the data what domain ?

    Thank you for your time.

    Regards,

    Mayk

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. 'Couldn't access Yahoo! Zimbra Desktop server"
    By chirag1 in forum Error Reports
    Replies: 37
    Last Post: 06-12-2011, 05:14 PM
  2. Message disappears between MTA and mailbox server
    By andrew_l in forum Administrators
    Replies: 12
    Last Post: 07-08-2010, 11:26 PM
  3. Emails bouncing with "Error Text: 401,'null'"
    By sholden in forum Zimbra Connector for Outlook
    Replies: 27
    Last Post: 08-20-2008, 04:59 PM
  4. Replies: 3
    Last Post: 07-19-2007, 02:00 AM
  5. Replies: 4
    Last Post: 08-10-2006, 10:55 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •