Results 1 to 7 of 7

Thread: Single-Copy Message Store and imapsync

  1. #1
    mlippold is offline Starter Member
    Join Date
    Jun 2007
    Location
    Portland, Oregon
    Posts
    2
    Rep Power
    8

    Default Single-Copy Message Store and imapsync

    Will transferring messages to Zimbra using imapsync create a single copy of messages sent to multiple users on the same server?

  2. #2
    dkarp is offline Zimbra Employee
    Join Date
    Aug 2005
    Posts
    1,433
    Rep Power
    12

    Default

    No, imapsync does not preserve the single-copy store.
    Bugzilla - Wiki - Downloads - Before posting... Search!

  3. #3
    mlippold is offline Starter Member
    Join Date
    Jun 2007
    Location
    Portland, Oregon
    Posts
    2
    Rep Power
    8

    Default

    I figured the MTA was responsible for this function. Is it a feature of Postfix?

    Is there a way to rebuild the single-message store?

    I am thinking about whipping up a Perl script to find messages in the message store that are the same and rebuilding the hard links. Do you see any issue with doing this?

  4. #4
    Rich Graves is offline Outstanding Member
    Join Date
    Jan 2007
    Location
    Minnesota
    Posts
    719
    Rep Power
    9

    Default

    It's a feature of the tomcat/mailboxd backend.

    Rather than writing your own, look at freedups file listing

    Unless the other end of the imapsync was also a single-instance store, you're unlikely to get much. The messages truly need to be *identical*, down to the final Received: header and delivery time.

  5. #5
    mikelcu is offline Senior Member
    Join Date
    Oct 2007
    Posts
    54
    Rep Power
    7

    Default

    If imapsync essentially breaks the single message store, what is the recommendation on migrating a reasonably large set of mailboxes with a large number of shared mails? Does the exchange migration utility preserve the single store?

    In my case, I've got ~100 mailboxes totaling more than 60GB that are chock-full of distribution-list and other mails stored on disk only once. If 25% of the mails are multi-instance-single-stored in the source mail server (ours could be more), that could throw off the storage calculations pretty badly...

    -Mike

  6. #6
    Rich Graves is offline Outstanding Member
    Join Date
    Jan 2007
    Location
    Minnesota
    Posts
    719
    Rep Power
    9

    Default

    Import wizards won't change the dedupe story on the Zimbra server end.

    If you've really got 25% shared mail to a large number of recipients and the distribution list messages contain attachments as big or bigger than personal messages, then indeed you do have a (very unusual) problem. I'm skeptical that it's as big a problem as you think, but I can think of a few workarounds.

    If the RFC822 message content that you can get out of Exchange is really *identical*, then you can probably de-duplicate retroactively with a program like hardlink or freedups. Migrate (portions of) two accounts and compare the "View Source" of each. I'd worry that Exchange or the import process would alter something in the headers. Try both imapsync and the import wizard to see if their behavior is different. If you can manage to end up with perfect duplicate blobs in different accounts, then pointing freedups at /opt/zimbra/store will recover disk space.

    If freedups won't work for you and you have a particular sender or subject that has filled everyone's mailbox with crap, you can tell imapsync to ignore it, make a shared mailbox including all of the distribution list's mail, and subscribe everyone to that shared mailbox.

    Storage technologies exist that virtualize and de-duplicate based on file-level or block-level content. They are mostly targeted at archive and backup, but NetApp advertises this for primary storage. Buzzphrases include "content-based addressing" and "global compression." These solutions tend to be far more expensive than would be justified for only 60GB, but if you're in the market for storage for other purposes, take a look.

    Last and least attractively, you could ask people to move bulk list mail to local PST folders, and only migrate what remains on the server.
    Last edited by Rich Graves; 08-02-2008 at 02:13 PM.

  7. #7
    stegbth is offline Special Member
    Join Date
    Sep 2008
    Posts
    134
    Rep Power
    6

    Default

    hello everybody,

    just to be sure.

    zimbra support's
    Single-Copy Message Store (or singleinstancestore as called on Cyrus imapd), if delivered eMails via smtp to zimbra
    does this work out of the box or must it get activated?
    especially if always_bcc is activated on Postfix?

    after moving a cyrus with imapsync to zimbra it is save to run freedups on /opt/zimbra/store

    greetings
    thomas
    Last edited by stegbth; 01-16-2009 at 09:44 AM.

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. Message store hard links
    By bdbell in forum Migration
    Replies: 0
    Last Post: 05-22-2007, 06:14 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •