Zimbra offers Open Source email server software and shared calendar for Linux and the Mac
Go Back   Zimbra :: Forums > Zimbra Collaboration Suite > Administrators

Welcome to the Zimbra :: Forums!
Welcome, if you would like to post a comment please register. We also encourage you to explore all things Zimbra with our team and members of the community.

Reply
 
LinkBack Thread Tools Search this Thread Display Modes
  #1 (permalink)  
Old 07-25-2008, 11:38 AM
Advanced Member
 
Posts: 193
Default [SOLVED] collapsing message volumes

I understand how one adds message volumes, but how do you ever remove them while maintaining the data on them? Lets say I start with one Xgb volume, add a second volume of Ygb and at a later point I want to replace both with a third volume of (X+Y+Z)gb.

Is there a way to migrate messages from one to the other or do I just have to play games with directory names and linking both volumes to directories on the new partition?
Reply With Quote
  #2 (permalink)  
Old 07-27-2008, 07:10 PM
Moderator
 
Posts: 6,236
Default

Solution1: Same Contents
If you want new store2 to hold the contents of the old store in entirely as well - just move the blobs with zimbra stopped first.
Then point the message1 path at /mount/store2. (Don't forget to check file permissions after the move.)
It may be better to use zmmailboxmove so you don't cross user 'folders' ie on the filesystem your now sharing dirs Bug 18850 - multiple "current" primary, secondary, index zmvolumes

Example: Volumes : Ajcody-Notes - Zimbra :: Wiki


Solution2: Old as subdirectory of new

(Having paths for both volume 1 & 2 point to disk 3 is common - better if you make it a subdirectory.)

If you've already created store2 and been using both for a while, just make the old store1 a subdirectory of the new store2 @ on the new storage device:
/mount/newstore2
/mount/store/oldstore1 (previously /opt/zimbra/store - change the path in zmvolume)




Solution3: Another method that can be used is updating the DB itself.

There's a volume_id column in the mail_item table in every mboxgroup database - after moving the blobs themselves you could run something like: mysql> for (N = 1 through 100) {UPDATE mboxgroupN.mail_item SET volume_id = IDofNew WHERE volume_id = IDofOld;}

The first N is talking about the mailbox group (id divided by 100) for which there's 100max groups.

Have a read of: Account mailbox database structure - Zimbra :: Wiki

If you wanted to poke around, some basic structure:
su - zimbra
mysql
show databases;
use mboxgroupN
show tables;
select * from mail_item where volume_id=1; (would get you all the items in volume 1)
That's a BIG query btw! So probably better if you selected by mailbox_id or id - narrow this down and try stuff on a test account or individual msg - better yet a test harness machine before you run any update/set/alter commands across every account on your production system.

CLI zmvolume - Zimbra :: Wiki
zmvolume -l
zmvolume -dc
mysql> select * from zimbra.volume;

MySQL commands: Pantz.org Technical Reference Site


Solution4: Use HSM to move the blobs after x time from store1 > store2.

---

re you saying LVM to size volume groups/pools by absorbing new physical disks/volumes to present to the system as one instance?

As far as Zimbra is concerned:
-Using LVM to expand /opt/zimbra, /opt/zimbra/store, or /opt/zimbra/index you don't have to do anything else.

-Haven't implemented LVM etc? Don't worry, you can still create 'expanding mail storage' even if not presented to the system as one big unit. If your creating an /opt/zimbra/store2, /mnt/device/zimbra/store2, or /mnt/device/zimbra/index2 that's separate - in order to get Zimbra to start putting blobs there you need to point the current message/index or volume there.

Just realize blobs are NOT moved in this process, you're only pointing at the location to store any NEW data. (See below explanation.)

Thus you shouldn't remove the old store, but alternatively if you want new store2 to hold the contents of the old store as well - just move the blobs first as in 1A.

Explanation:

Each Zimbra mailbox server is configured with one current index & message (NE can add secondary HSM volumes).

When a new message is delivered or created, the message is saved in the current message volume. Additional message volumes can be created, but only one is configured as the current volume where new messages are stored. When the volume is full, you can configure a new current message volume. The current message volume receives all new messages. New messages are never stored in the previous volume. A current volume cannot be deleted. If there are accounts with messages referencing a volume it should not be deleted until you point the volume path to wherever you've now moved the items.

Each mailbox is assigned to a permanent index directory on the current index volume. When an account is created, the current index volume is automatically defined for the account. You cannot change which index volume the account is assigned. As volumes become full, you can create a new current index volume for new accounts. When a new index volume is added as current, the older index volume is no longer assigned new accounts. Index volumes not marked current are still actively in use as the index volumes for accounts assigned to them. Any index volume that is referenced by a mailbox as it's index volume cannot be deleted.

The Network Edition has HSM (Hierarchical Storage Management) which involves moving data to other storage locations automatically after x amount of time. Messages and attachments are moved from a primary volume to the current secondary volume based on the age of the message; completely transparent to the user.

For instance: After 30 days messages move from your fast (and usually more expensive) SCSI disks, to a SATA array so that you can keep storage costs down. But you could also use it to move between your 64GB & 320GB volumes if they're separate.

Bug 18850 - multiple "current" primary, secondary, index zmvolumes
Bug 18720 - Add support for more than one current secondary storage volume in HSM
Bug 32719 - Add ability for admins to configure what blobs get migrated with HSM

Bug 23472 - Ability to move/consolidate messages from one message store to another.
Bug 35142 - volume migration tool

If your merging two systems it may be better to use zmmailboxmove so you don't cross user 'folders' ie on the filesystem will go into the 'current'/ you don't want to share user level dirs. To solve this we really should append a zimbraId string to each users blob folder path (after the 100 group number), or even make it entirely like we do for backups.

Last edited by mmorse; 04-28-2010 at 02:47 PM..
Reply With Quote
  #3 (permalink)  
Old 07-28-2008, 01:20 AM
Moderator
 
Posts: 2,207
Default

AFAIK, you can also :
. stop ZCS
. move the messages by hand to a new filesystem/volume
. start ZCS (in a way it does not receive mails and users can not connect to it)
. change the path of the volume in the webadmin or CLI (you keep the same name but change the path)
. restart ZCS (in a normal way)

This method allows you not to do changes in the database.
Reply With Quote
  #4 (permalink)  
Old 09-20-2008, 05:29 PM
Intermediate Member
 
Posts: 22
Default

Quote:
Originally Posted by Klug View Post
AFAIK, you can also :
. stop ZCS
. move the messages by hand to a new filesystem/volume
. start ZCS (in a way it does not receive mails and users can not connect to it)
. change the path of the volume in the webadmin or CLI (you keep the same name but change the path)
. restart ZCS (in a normal way)

This method allows you not to do changes in the database.
Hi Klug,

thank you for this interesting pointer.. Is this method tested ?
Can i move the complete /store folder to a new location or must i create a new one and move it ?
i presume changing the path of the volume is done with zmvolume -e ?

I'm asking this because i'm about te have about 100 extra users on the server, and want some extra space on some external storage.

When i create a new volume , the new volume will be the current volume and so receive all the new incoming mail.

Could it be possible to define which volume holds the data what domain ?

Thank you for your time.

Regards,

Mayk
Reply With Quote
Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes


Similar Threads

Why Join?

Registering let's you ask questions, makes it easier to search, displays any files attached to posts, and notifies you about replies.

blog.zimbra.com




 

SEO by vBSEO ©2011, Crawlability, Inc.