Results 1 to 10 of 10

Thread: lmtp - Not delivering message with duplicate Message-ID

  1. #1
    mps
    mps is offline Member
    Join Date
    Feb 2007
    Location
    Orlandia, Sao Paulo, Brazil
    Posts
    11
    Rep Power
    8

    Default lmtp - Not delivering message with duplicate Message-ID

    Hello,

    I'm taking a look to the mailbox.log and I'm seeing this message:
    "lmtp - Not delivering message with duplicate Message-ID"

    The incoming message is for multiple recipients.

    The users are not receiving this message? Or is there some internal control for this in Zimbra?

    Version of Zimbra: 5.0.4_GA_2101.DEBIAN4.0

    Thanks in advance.
    Last edited by mps; 05-02-2008 at 04:29 AM. Reason: Adding Zimbra version

  2. #2
    mmorse's Avatar
    mmorse is offline Moderator
    Join Date
    May 2006
    Location
    USA
    Posts
    6,242
    Rep Power
    21

    Default

    When LMTP receives the same message for the same user twice in a short period of time, it drops the second copy - this all takes place in memory. The server will keep that many recipient/Message-ID pairs in a FIFO map (first in, first out) dropping any duplicates it sees. Though I don't think you need to, it's possible to disable this by setting zimbraMessageIdDedupeCacheSize to 0 in the global LDAP config. Useful when you're using zmlmtpinject over and over & want it to actually make more copies. For normal operation default is 3000.

  3. #3
    mps
    mps is offline Member
    Join Date
    Feb 2007
    Location
    Orlandia, Sao Paulo, Brazil
    Posts
    11
    Rep Power
    8

    Default

    mmorse, thanks for the explanation!

  4. #4
    sfilipov is offline Starter Member
    Join Date
    Jun 2008
    Posts
    2
    Rep Power
    7

    Default

    Hi,

    I have an issue with messages which are not coming and after some investigation I've found what is the problem.
    Fisrt at 13:55 user jatanasova have sent message with subject "info profile diageo" to user mbobeva. Message was received normally.
    Then at 14:07 and 14:25 user jatanasova have sent message with subject "Fwd: info profile diageo" to user mbobeva, but messages didn't come at all. Mailbox.log says: "lmtp - Not delivering message with duplicate Message-ID".
    Then at 14:40 user jatanasova have sent message with subject "Fwd: info profile diageo" to distribution list in which mbobeva is participating, but again message was stopped with the same error.

    Is this normal, and why is this happening - messages are different.

    And from where/how can I change zimbraMessageIdDedupeCacheSize to 0. And did this hide some risks?

    Thanks in advance.
    Bellow is the log:


    2008-06-30 13:55:04,849 INFO [LmtpServer-4952] [ip=212.72.217.102;] lmtp - Delivering message: size=25967 bytes, nrcpts=1, sender=jatanasova@manager.bg, msgid=<!&!AAAAAAAAAAAYAAAAAAAAAHmF2r0UnVVHgGaOP3kP tZ7CgAAAEAAAAPN8/SQ5MydJnMuBTWMxhf4BAAAAAA==@manager.bg>
    2008-06-30 13:55:04,883 INFO [LmtpServer-4952] [name=mbobeva@manager.bg;mid=68;] mailop - Adding Message: id=3174, Message-ID=<!&!AAAAAAAAAAAYAAAAAAAAAHmF2r0UnVVHgGaOP3kPtZ7 CgAAAEAAAAPN8/SQ5MydJnMuBTWMxhf4BAAAAAA==@manager.bg>, parentId=-1, folderId=2, folderName=Inbox.

    2008-06-30 14:07:05,587 INFO [LmtpServer-4965] [ip=212.72.217.102;] lmtp - Delivering message: size=34200 bytes, nrcpts=1, sender=jatanasova@manager.bg, msgid=<!&!AAAAAAAAAAAYAAAAAAAAAHmF2r0UnVVHgGaOP3kP tZ7CgAAAEAAAAPN8/SQ5MydJnMuBTWMxhf4BAAAAAA==@manager.bg>
    2008-06-30 14:07:05,588 INFO [LmtpServer-4965] [name=mbobeva@manager.bg;mid=68;] lmtp - Not delivering message with duplicate Message-ID <!&!AAAAAAAAAAAYAAAAAAAAAHmF2r0UnVVHgGaOP3kPtZ7CgA AAEAAAAPN8/SQ5MydJnMuBTWMxhf4BAAAAAA==@manager.bg>

    2008-06-30 14:25:21,555 INFO [LmtpServer-4993] [ip=212.72.217.102;] lmtp - Delivering message: size=35543 bytes, nrcpts=1, sender=jatanasova@manager.bg, msgid=<!&!AAAAAAAAAAAYAAAAAAAAAHmF2r0UnVVHgGaOP3kP tZ7CgAAAEAAAAPN8/SQ5MydJnMuBTWMxhf4BAAAAAA==@manager.bg>
    2008-06-30 14:25:21,556 INFO [LmtpServer-4993] [name=mbobeva@manager.bg;mid=68;] lmtp - Not delivering message with duplicate Message-ID <!&!AAAAAAAAAAAYAAAAAAAAAHmF2r0UnVVHgGaOP3kPtZ7CgA AAEAAAAPN8/SQ5MydJnMuBTWMxhf4BAAAAAA==@manager.bg>

    2008-06-30 14:40:45,608 INFO [LmtpServer-5018] [ip=212.72.217.102;] lmtp - Delivering message: size=36629 bytes, nrcpts=9, sender=jatanasova@manager.bg, msgid=<!&!AAAAAAAAAAAYAAAAAAAAAHmF2r0UnVVHgGaOP3kP tZ7CgAAAEAAAAPN8/SQ5MydJnMuBTWMxhf4BAAAAAA==@manager.bg>
    2008-06-30 14:40:45,635 INFO [LmtpServer-5018] [name=dlukanova@manager.bg;mid=69;] mailop - Adding Message: id=2307, Message-ID=<!&!AAAAAAAAAAAYAAAAAAAAAHmF2r0UnVVHgGaOP3kPtZ7 CgAAAEAAAAPN8/SQ5MydJnMuBTWMxhf4BAAAAAA==@manager.bg>, parentId=-1, folderId=2, folderName=Inbox.
    2008-06-30 14:40:45,679 INFO [LmtpServer-5018] [name=ggoranov@manager.bg;mid=70;] mailop - Adding Message: id=2077, Message-ID=<!&!AAAAAAAAAAAYAAAAAAAAAHmF2r0UnVVHgGaOP3kPtZ7 CgAAAEAAAAPN8/SQ5MydJnMuBTWMxhf4BAAAAAA==@manager.bg>, parentId=-1, folderId=2, folderName=Inbox.
    2008-06-30 14:40:45,692 INFO [LmtpServer-5018] [name=ghristova@manager.bg;mid=71;] mailop - Adding Message: id=2108, Message-ID=<!&!AAAAAAAAAAAYAAAAAAAAAHmF2r0UnVVHgGaOP3kPtZ7 CgAAAEAAAAPN8/SQ5MydJnMuBTWMxhf4BAAAAAA==@manager.bg>, parentId=-1, folderId=2, folderName=Inbox.
    2008-06-30 14:40:45,716 INFO [LmtpServer-5018] [name=iivanova@manager.bg;mid=72;] mailop - Adding Message: id=2133, Message-ID=<!&!AAAAAAAAAAAYAAAAAAAAAHmF2r0UnVVHgGaOP3kPtZ7 CgAAAEAAAAPN8/SQ5MydJnMuBTWMxhf4BAAAAAA==@manager.bg>, parentId=-1, folderId=2, folderName=Inbox.
    2008-06-30 14:40:45,735 INFO [LmtpServer-5018] [name=jatanasova@manager.bg;mid=66;] mailop - Adding Message: id=3343, Message-ID=<!&!AAAAAAAAAAAYAAAAAAAAAHmF2r0UnVVHgGaOP3kPtZ7 CgAAAEAAAAPN8/SQ5MydJnMuBTWMxhf4BAAAAAA==@manager.bg>, parentId=-1, folderId=2, folderName=Inbox.
    2008-06-30 14:40:45,744 INFO [LmtpServer-5018] [name=mbobeva@manager.bg;mid=68;] lmtp - Not delivering message with duplicate Message-ID <!&!AAAAAAAAAAAYAAAAAAAAAHmF2r0UnVVHgGaOP3kPtZ7CgA AAEAAAAPN8/SQ5MydJnMuBTWMxhf4BAAAAAA==@manager.bg>
    2008-06-30 14:40:45,746 INFO [LmtpServer-5018] [name=mmarinova@manager.bg;mid=73;] mailop - Adding Message: id=2153, Message-ID=<!&!AAAAAAAAAAAYAAAAAAAAAHmF2r0UnVVHgGaOP3kPtZ7 CgAAAEAAAAPN8/SQ5MydJnMuBTWMxhf4BAAAAAA==@manager.bg>, parentId=-1, folderId=2, folderName=Inbox.
    2008-06-30 14:40:45,876 INFO [LmtpServer-5018] [name=sfilipov@manager.bg;mid=48;] mailop - Adding Message: id=1798, Message-ID=<!&!AAAAAAAAAAAYAAAAAAAAAHmF2r0UnVVHgGaOP3kPtZ7 CgAAAEAAAAPN8/SQ5MydJnMuBTWMxhf4BAAAAAA==@manager.bg>, parentId=-1, folderId=1494, folderName=ADV.
    2008-06-30 14:40:45,895 INFO [LmtpServer-5018] [name=sgeorgieva@manager.bg;mid=74;] mailop - Adding Message: id=2206, Message-ID=<!&!AAAAAAAAAAAYAAAAAAAAAHmF2r0UnVVHgGaOP3kPtZ7 CgAAAEAAAAPN8/SQ5MydJnMuBTWMxhf4BAAAAAA==@manager.bg>, parentId=-1, folderId=2, folderName=Inbox.

  5. #5
    fowler is offline Trained Alumni
    Join Date
    Jan 2009
    Location
    Fresno
    Posts
    31
    Rep Power
    6

    Default

    Quote Originally Posted by mmorse View Post
    When LMTP receives the same message for the same user twice in a short period of time, it drops the second copy
    ...
    it's possible to disable this by setting zimbraMessageIdDedupeCacheSize to 0 in the global LDAP config.
    Code:
    zmprov gcf zimbraMessageIdDedupeCacheSize
    3000
    zmprov mcf zimbraMessageIdDedupeCacheSize 0
    zmprov gcf zimbraMessageIdDedupeCacheSize
    0
    NOTE: Even after flushing the config cache (zmprov fc config), the change was not reflected. It appears a restart is necessary for the change to take affect.

    See also: Flushing LDAP Cache

  6. #6
    liuk is offline Intermediate Member
    Join Date
    Apr 2009
    Location
    Italy - Genova
    Posts
    21
    Rep Power
    6

    Default

    Hello Mike,
    we are facing similare issues here with Clients using Outlook 2010. For example it the user forward the same email again changing the attachment the message is discarded for the recipient.
    Since from what I can read on RFC 2822 :
    the existence of a message id is an optional (SHOULD have) requirement, but its uniqueness if it is supplied is necessary (MUST be)
    I suppose that the fault is from the Outlook client.

    Now my other questions are: what is the purpose of this caching ? And what can be the side effects of setting it to ZERO ?

    Thanks!
    Luca

  7. #7
    mattg1 is offline Active Member
    Join Date
    Jun 2009
    Posts
    31
    Rep Power
    6

    Default

    Hi... We just encountered this issue here too. In fact, it has been going on for a long time, but was just noticed now. Users use Outlook templates to send messages via ZCO. The messages from that template always have the same Message-ID. Apparently this is a known Outlook problem for a long time. Awesome. Anyway, I guess changing zimbraMessageIdDedupeCacheSize is a workaround, but may have unintended consequences in the case of mail loops and such. (Perhaps merely reducing the value rather than zeroing it would help the problem, while still providing some loop protection.)

    Another option might be for ZCO to remove the Message-ID header (which is optional from the MUA's end) so that the Zimbra server can generate a new unique one. Or, even configure the Zimbra server to ignore client-generated Message-IDs on submitted emails. I don't know if those would violate some RFC, though.

    Any other ideas besides tweaking zimbraMessageIdDedupeCacheSize ? (Or getting rid of Outlook?)

  8. #8
    jerryipe is offline Starter Member
    Join Date
    May 2013
    Posts
    1
    Rep Power
    2

    Default Meaning of zimbraMessageIdDedupeCacheSize

    I have read the above post, i am also facing the same issue mentioned above,
    Any idea about zimbraMessageIdDedupeCacheSize, what does the value 3000 means.
    Any one please give an idea.

    Regards,
    Jerry I. George


    Quote Originally Posted by mattg1 View Post
    Hi... We just encountered this issue here too. In fact, it has been going on for a long time, but was just noticed now. Users use Outlook templates to send messages via ZCO. The messages from that template always have the same Message-ID. Apparently this is a known Outlook problem for a long time. Awesome. Anyway, I guess changing zimbraMessageIdDedupeCacheSize is a workaround, but may have unintended consequences in the case of mail loops and such. (Perhaps merely reducing the value rather than zeroing it would help the problem, while still providing some loop protection.)

    Another option might be for ZCO to remove the Message-ID header (which is optional from the MUA's end) so that the Zimbra server can generate a new unique one. Or, even configure the Zimbra server to ignore client-generated Message-IDs on submitted emails. I don't know if those would violate some RFC, though.

    Any other ideas besides tweaking zimbraMessageIdDedupeCacheSize ? (Or getting rid of Outlook?)

  9. #9
    liuk is offline Intermediate Member
    Join Date
    Apr 2009
    Location
    Italy - Genova
    Posts
    21
    Rep Power
    6

    Default

    In the release notes of 7.1.4:
    Global LDAP attrib zimbraMessageIdDedupeCacheTimeout for LMTP de-dupe cache to workaround clients incorrectly using the same ID for different messages. If non-zero, zimbraMessageIdDedupeCacheSize limit is ignored. (44564)

    Did you try to first reduce the zimbraMessageIdDedupeCacheSize to a small value ?

    If you set it to zero users will receive duplicate e-mails under certain conditions, but there are no other side effects AFAIK.

    HTH
    Luca

  10. #10
    mattg1 is offline Active Member
    Join Date
    Jun 2009
    Posts
    31
    Rep Power
    6

    Default

    Quote Originally Posted by jerryipe View Post
    Any idea about zimbraMessageIdDedupeCacheSize, what does the value 3000 means.
    The way I understand it, it means that Zimbra will remember the last 3000 Message-IDs it sees, and prevent delivery of any "duplicates" of those 3000 messages. It's up to you to figure out how much email your server sees, and adjust that value to estimate how long in seconds this might be. On my server, I had to set it to 0 to allow Outlook users to do what they wanted reliably.

    Quote Originally Posted by liuk View Post
    Global LDAP attrib zimbraMessageIdDedupeCacheTimeout for LMTP de-dupe cache to workaround clients incorrectly using the same ID for different messages. If non-zero, zimbraMessageIdDedupeCacheSize limit is ignored. (44564)
    Thanks.... I will have to look into zimbraMessageIdDedupeCacheTimeout to see if it might be an alternative for me.

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. Emails bouncing with "Error Text: 401,'null'"
    By sholden in forum Zimbra Connector for Outlook
    Replies: 27
    Last Post: 08-20-2008, 04:59 PM
  2. Replies: 3
    Last Post: 07-19-2007, 02:00 AM
  3. zimbra generates occasional duplicate message
    By ljm in forum Administrators
    Replies: 0
    Last Post: 01-15-2007, 09:43 PM
  4. Replies: 42
    Last Post: 08-11-2006, 09:50 AM
  5. Replies: 4
    Last Post: 08-10-2006, 10:55 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •