lmtp - Not delivering message with duplicate Message-ID

[mps]

Hello,

I'm taking a look to the mailbox.log and I'm seeing this message:
"lmtp - Not delivering message with duplicate Message-ID"

The incoming message is for multiple recipients.

The users are not receiving this message? Or is there some internal control for this in Zimbra?

Version of Zimbra: 5.0.4_GA_2101.DEBIAN4.0

Thanks in advance.

[mmorse]

When LMTP receives the same message for the same user twice in a short period of time, it drops the second copy - this all takes place in memory. The server will keep that many recipient/Message-ID pairs in a FIFO map (first in, first out) dropping any duplicates it sees. Though I don't think you need to, it's possible to disable this by setting zimbraMessageIdDedupeCacheSize to 0 in the global LDAP config. Useful when you're using zmlmtpinject over and over & want it to actually make more copies. For normal operation default is 3000.

[mps]

mmorse, thanks for the explanation!

[sfilipov]

Hi,

I have an issue with messages which are not coming and after some investigation I've found what is the problem.
Fisrt at 13:55 user jatanasova have sent message with subject "info profile diageo" to user mbobeva. Message was received normally.
Then at 14:07 and 14:25 user jatanasova have sent message with subject "Fwd: info profile diageo" to user mbobeva, but messages didn't come at all. Mailbox.log says: "lmtp - Not delivering message with duplicate Message-ID".
Then at 14:40 user jatanasova have sent message with subject "Fwd: info profile diageo" to distribution list in which mbobeva is participating, but again message was stopped with the same error.

Is this normal, and why is this happening - messages are different.

And from where/how can I change zimbraMessageIdDedupeCacheSize to 0. And did this hide some risks?

Thanks in advance.
Bellow is the log:


2008-06-30 13:55:04,849 INFO [LmtpServer-4952] [ip=212.72.217.102;] lmtp - Delivering message: size=25967 bytes, nrcpts=1, sender=jatanasova@manager.bg, msgid=<!&!AAAAAAAAAAAYAAAAAAAAAHmF2r0UnVVHgGaOP3kP tZ7CgAAAEAAAAPN8/SQ5MydJnMuBTWMxhf4BAAAAAA==@manager.bg>
2008-06-30 13:55:04,883 INFO [LmtpServer-4952] [name=mbobeva@manager.bg;mid=68;] mailop - Adding Message: id=3174, Message-ID=<!&!AAAAAAAAAAAYAAAAAAAAAHmF2r0UnVVHgGaOP3kPtZ7 CgAAAEAAAAPN8/SQ5MydJnMuBTWMxhf4BAAAAAA==@manager.bg>, parentId=-1, folderId=2, folderName=Inbox.

2008-06-30 14:07:05,587 INFO [LmtpServer-4965] [ip=212.72.217.102;] lmtp - Delivering message: size=34200 bytes, nrcpts=1, sender=jatanasova@manager.bg, msgid=<!&!AAAAAAAAAAAYAAAAAAAAAHmF2r0UnVVHgGaOP3kP tZ7CgAAAEAAAAPN8/SQ5MydJnMuBTWMxhf4BAAAAAA==@manager.bg>
2008-06-30 14:07:05,588 INFO [LmtpServer-4965] [name=mbobeva@manager.bg;mid=68;] lmtp - Not delivering message with duplicate Message-ID <!&!AAAAAAAAAAAYAAAAAAAAAHmF2r0UnVVHgGaOP3kPtZ7CgA AAEAAAAPN8/SQ5MydJnMuBTWMxhf4BAAAAAA==@manager.bg>

2008-06-30 14:25:21,555 INFO [LmtpServer-4993] [ip=212.72.217.102;] lmtp - Delivering message: size=35543 bytes, nrcpts=1, sender=jatanasova@manager.bg, msgid=<!&!AAAAAAAAAAAYAAAAAAAAAHmF2r0UnVVHgGaOP3kP tZ7CgAAAEAAAAPN8/SQ5MydJnMuBTWMxhf4BAAAAAA==@manager.bg>
2008-06-30 14:25:21,556 INFO [LmtpServer-4993] [name=mbobeva@manager.bg;mid=68;] lmtp - Not delivering message with duplicate Message-ID <!&!AAAAAAAAAAAYAAAAAAAAAHmF2r0UnVVHgGaOP3kPtZ7CgA AAEAAAAPN8/SQ5MydJnMuBTWMxhf4BAAAAAA==@manager.bg>

2008-06-30 14:40:45,608 INFO [LmtpServer-5018] [ip=212.72.217.102;] lmtp - Delivering message: size=36629 bytes, nrcpts=9, sender=jatanasova@manager.bg, msgid=<!&!AAAAAAAAAAAYAAAAAAAAAHmF2r0UnVVHgGaOP3kP tZ7CgAAAEAAAAPN8/SQ5MydJnMuBTWMxhf4BAAAAAA==@manager.bg>
2008-06-30 14:40:45,635 INFO [LmtpServer-5018] [name=dlukanova@manager.bg;mid=69;] mailop - Adding Message: id=2307, Message-ID=<!&!AAAAAAAAAAAYAAAAAAAAAHmF2r0UnVVHgGaOP3kPtZ7 CgAAAEAAAAPN8/SQ5MydJnMuBTWMxhf4BAAAAAA==@manager.bg>, parentId=-1, folderId=2, folderName=Inbox.
2008-06-30 14:40:45,679 INFO [LmtpServer-5018] [name=ggoranov@manager.bg;mid=70;] mailop - Adding Message: id=2077, Message-ID=<!&!AAAAAAAAAAAYAAAAAAAAAHmF2r0UnVVHgGaOP3kPtZ7 CgAAAEAAAAPN8/SQ5MydJnMuBTWMxhf4BAAAAAA==@manager.bg>, parentId=-1, folderId=2, folderName=Inbox.
2008-06-30 14:40:45,692 INFO [LmtpServer-5018] [name=ghristova@manager.bg;mid=71;] mailop - Adding Message: id=2108, Message-ID=<!&!AAAAAAAAAAAYAAAAAAAAAHmF2r0UnVVHgGaOP3kPtZ7 CgAAAEAAAAPN8/SQ5MydJnMuBTWMxhf4BAAAAAA==@manager.bg>, parentId=-1, folderId=2, folderName=Inbox.
2008-06-30 14:40:45,716 INFO [LmtpServer-5018] [name=iivanova@manager.bg;mid=72;] mailop - Adding Message: id=2133, Message-ID=<!&!AAAAAAAAAAAYAAAAAAAAAHmF2r0UnVVHgGaOP3kPtZ7 CgAAAEAAAAPN8/SQ5MydJnMuBTWMxhf4BAAAAAA==@manager.bg>, parentId=-1, folderId=2, folderName=Inbox.
2008-06-30 14:40:45,735 INFO [LmtpServer-5018] [name=jatanasova@manager.bg;mid=66;] mailop - Adding Message: id=3343, Message-ID=<!&!AAAAAAAAAAAYAAAAAAAAAHmF2r0UnVVHgGaOP3kPtZ7 CgAAAEAAAAPN8/SQ5MydJnMuBTWMxhf4BAAAAAA==@manager.bg>, parentId=-1, folderId=2, folderName=Inbox.
2008-06-30 14:40:45,744 INFO [LmtpServer-5018] [name=mbobeva@manager.bg;mid=68;] lmtp - Not delivering message with duplicate Message-ID <!&!AAAAAAAAAAAYAAAAAAAAAHmF2r0UnVVHgGaOP3kPtZ7CgA AAEAAAAPN8/SQ5MydJnMuBTWMxhf4BAAAAAA==@manager.bg>
2008-06-30 14:40:45,746 INFO [LmtpServer-5018] [name=mmarinova@manager.bg;mid=73;] mailop - Adding Message: id=2153, Message-ID=<!&!AAAAAAAAAAAYAAAAAAAAAHmF2r0UnVVHgGaOP3kPtZ7 CgAAAEAAAAPN8/SQ5MydJnMuBTWMxhf4BAAAAAA==@manager.bg>, parentId=-1, folderId=2, folderName=Inbox.
2008-06-30 14:40:45,876 INFO [LmtpServer-5018] [name=sfilipov@manager.bg;mid=48;] mailop - Adding Message: id=1798, Message-ID=<!&!AAAAAAAAAAAYAAAAAAAAAHmF2r0UnVVHgGaOP3kPtZ7 CgAAAEAAAAPN8/SQ5MydJnMuBTWMxhf4BAAAAAA==@manager.bg>, parentId=-1, folderId=1494, folderName=ADV.
2008-06-30 14:40:45,895 INFO [LmtpServer-5018] [name=sgeorgieva@manager.bg;mid=74;] mailop - Adding Message: id=2206, Message-ID=<!&!AAAAAAAAAAAYAAAAAAAAAHmF2r0UnVVHgGaOP3kPtZ7 CgAAAEAAAAPN8/SQ5MydJnMuBTWMxhf4BAAAAAA==@manager.bg>, parentId=-1, folderId=2, folderName=Inbox.

[fowler]

Originally Posted by mmorse

When LMTP receives the same message for the same user twice in a short period of time, it drops the second copy
...
it's possible to disable this by setting zimbraMessageIdDedupeCacheSize to 0 in the global LDAP config.

zmprov gcf zimbraMessageIdDedupeCacheSize
3000
zmprov mcf zimbraMessageIdDedupeCacheSize 0
zmprov gcf zimbraMessageIdDedupeCacheSize
0

NOTE: Even after flushing the config cache (zmprov fc config), the change was not reflected. It appears a restart is necessary for the change to take affect.

See also: Flushing LDAP Cache