Well for starters check the logs during your tests - by default only 1 away message is sent per incoming message address.
You could try:
zmprov ma
user@domain.com zimbraPrefOutOfOfficeCacheDuration 0
(you could also modify it on the COS)
Setting to 0 would send an out-of-office notification in response to every email received. (You want this while testing-but it's generally not a good idea, as your sending responses to spam etc...)
Plus, if you have user A with an away message up, user B sends an email, and immediately puts up their away message (with the attribute set to 0) you could technically get a loop. Therefore at a minimum use 1 day:
zmprov ma
user@domain.com zimbraPrefOutOfOfficeCacheDuration 1d
Thus duplicate away messages will not be sent to the same user in the given interval of 1 day. On day 2, if they send that account an email, they would again get back one notification, then no notifications for another day.
I prefer 3d or every week 5/7d depending upon how long the user is gone. ie: If their gone a month-then every week, (this is what I set the COS to) unless they specifically ask for 3 days.
You also might be interested in:
zimbraPrefOutOfOfficeDirectAddress
zimbraPrefOutOfOfficeFromDate
zimbraPrefOutOfOfficeReply
zimbraPrefOutOfOfficeReplyEnabled
zimbraPrefOutOfOfficeUntilDate