Archiving feature requests

[chh]

Hi,

I think that fully integrated archiving is one of the most important features that speak for Zimbra compared to other solutions.
Still it lacks some functions that would be nice or even important.

i would like to gather all these features here, some might already be in bugzilla and others could be added if the feedback is positive here.

So here is what I would like to see:

- Existing mails are cloned into the archive when enabled (suggested here Adding existing accounts to archive)

- Read access on a user level (RFE here Bug 18636 - Archiving: give end-user read-only view of their archive)

- possibility to write archive to a seperate store (cheaper disks) or to use a different HSM schedule, either on COS or domain level

- easy way to use a seperate backup schedule/configuration for archive accounts

- possibility to set a retention time for the archives.

- some easy way to delete older mails from the regular accounts and perhaps even replacing those by links to the archive

- single instancing (if not already present) at least within the archive mailboxes

Now please give your feedbacks on the ideas and add your own.

Thanks

Christian

[mmorse]

Those all sound excellent-thankyou for compiling them!
A few notes:

Originally Posted by chh

Read access on a user level (RFE here Bug 18636 - Archiving: give end-user read-only view of their archive)

As long as it's read only, then they could forward emails to their regular account when they want it restored. (but if you plan to have different archive message lifetimes, you don't want them depending on it)

some easy way to delete older mails from the regular accounts and perhaps even replacing those by links to the archive

-This would totally change the concept of archiving & discovery. The idea is to use HSM to offload older data to cheaper disk so they wouldn't have to use a complicated scheme like groupwise uses, instead they just make their own archive/save folder within their regular account.
-There's an RFE for different HSM location/schedule per folders, which would work to the concept you desire, but that's nothing to do with archiving for compliance purposes. Bug 9178 - Folder based HSM/aging

single instancing (if not already present) at least within the archive mailboxes

-HSM & single instance storage already takes place (per mailstore-so if the archives are on another mailstore, sry-this won't change for a multitude of performance reasons, ie you have 2 geographically separated offices...it cuts down on your internet traffic)

possibility to write archive to a seperate store (cheaper disks) or to use a different HSM schedule, either on COS or domain level

In the mean time, you would have to put the archive accounts on a separate mailstore with a different HSM time/location. (nothing like having a separate archive box right?)
You move those that you want hsm set to 30days (the regular accounts) to mailstore1 & those that you want hsm to kick in after just 5 days (archive accounts) to mailstore2.

But that get's expensive so...
-a separate place to store archives should really be an RFE in general for "the ability to designate user's storage (and HSM storage) to separate volumes within the same mailstore". (that is gonna get extremely complex on the mind)

easy way to use a seperate backup schedule/configuration for archive accounts

Builds on the same idea, because no one likes wasting space:
-A general RFE "ability to select which accounts go into the standard backup schedule via the admin console gui" (obviously picking accounts can already be done during a cli zmbackup)
So you would have:
a) All the regular accounts in the regular scheduled backup
b) Then you run a manual backup when wanted and select only the archive accounts.

Course you could always open an RFE for: 'select users for multiple different backup locations and corresponding different schedules' but the concept of full backups would change.

possibility to set a retention time for the archives.

-probably could be achieved with email message lifetime, which is the number of days a message can remain in any folder before it is automatically purged. The default is 0; email messages are not deleted.

[mmorse]

Executive summary:
Could you check bugzilla (too see if these have been already made) & if not create new ones? -be sure to post links back here so we know where to vote!

1. When turning on archiving with zmarchiveconfig, existing emails in account should be cloned as part of the process. (so you don't have to run imapsync/depend on PST import)

2. The ability to select which accounts go into the standard backup schedule via the admin console GUI (search first)

And so people can keep track-there's the already made:
Bug 18636 - Archiving: give end-user read-only view of their archive
&
Bug 9178 - Folder based HSM/aging

[chh]

It seems we are thinking in the same line.
Unfortunately Quoting does not work well so ehre are my responses:

re read acces to archive:
true, I see it as some kind of "self service restore" or the chance to reduce active mail volume.

re old mail and links to archive.
I like the idea of Enterprise Vault and the likes to remove mails from the main mailbox and replace them by links to the archive, so it might not be so important with Zimbra.

store for archive
my idea is to keep the archive on lower speed disks as they are not accessed that much, so they need not be on my 15k RAID 1 volumes.
So either we should be able to decide on some level to use a specific mail store.
User level might be too complicated, and not really needed, I guess. But COS or domain level might work out. So that the .archive domain uses the slower disks.

re Backup
While I like the idea to have a month or so of backup available for online restore, I think this is not needed for the archive.
Plus, when moved to tape, the tape will surely not be the same that the life data should be on.
I know that it is possible to select the accounts to backup. But this way it is very hard to seperate lets say a 100 accounts and 100+ archive-accounts and configure seperate backups for them. It should be possibe to backup all users of domain xxx with this schedule and yyy accounts with another.


I will wait opening the RFEs until I get some feedback here to mix it with my ideas

[chh]

I will check bugzilla, and open the RFEs but want to get some feedback here first.

I am not quite sure what you mean with this last one, and if it matches what I mean, I hope I made my intention more clear in my previous post.

And there's the already made:
2. Bug 18636 - Archiving: give end-user read-only view of their archive

Found that one and listed it already above.

I searched for archive in bugzilla and did not find any other of our points yet.

[mmorse]

user read access-lol-you must have just missed my edit:

Originally Posted by mmorse

As long as it's read only, then they could forward emails to their regular account when they want it restored. (but if you plan to have different archive lifetimes, you don't want them depending on it)

I was merely adding 18636 at the bottom so people were aware of it (there-edited the prior post so people understand that it's already open)

You can always open new ones-if duplicates are found they'll be tagged as such. -When you make one, might want to make the url pointer to this thread, so they understand what's being asked.

I'd say go ahead and open these two:

1. When turning on archiving with zmarchiveconfig, existing emails in account should be cloned as part of the process. (so you don't have to run imapsync/depend on PST import)

2. The ability to select which accounts go into the standard backup schedule via the admin console GUI (search first)

There's obviously has to be serious thought that has to go into the more complex storage related ones. -I don't see those happening anytime soon

Also, realize original point of ZAD was for compliance with the ability to keep it around for a set lifetime & make it easily searchable.
So if your still looking for having an archive folder always in the user list like a 'keep' so it's not confused with the other archiving for compliance see: Bug 9178 - Folder based HSM/aging
-which got incorporated into Bug 6542 - Zimbra Message Store & HSM Aging Policies (yes that bug has a bunch of different things in it)