sending an email (size: 6K) to 16000 recipients

[tiger2000]

Dear All,

Just hope to get some experience if someone occurs same situation as mine :

We're testing a ZCS (5.0.13 version Open Source Edition) on a Dell PE1950, 8G RAM, 2CPU (quad-core), external storage with Qlogic HBA (4Gbps).


When we test to send an email with 6K size to total 16000 local recipients, we found it almost spends 30 minutes when users can really see such email to enter their Inbox. and in the meantime, if someone sends emails from external site (e.g. @gmail.com ==> our site), it will be put into queue and be delivered after these 16000 delivery finished.

my questions are :

1. is there any system parameter which we can tune to speed up the delivery ?

2. if sender choose the "priority" and set it "Low" , what will happen? will system delays to this email to 16000 recipients , and will proceed other normal emails first ?

3. one email with 6K size to 16000 recipients in 30 minutes. does this number make sense? any knows how about in other email system?

Thanks.

[veronica]

You can adjust :-

default_destination_concurrency_limit and lmtp_destination_concurrency_limit

[veronica]

Priority doesn't mean messages will be handled first by zimbra MTA. Priority is end user feature and should not be confused with MTA queue scheduling. All messages are treated equal unles seperate queue is configured for a domain. zimbra MTA round robins between deferred queue and incoming queue.

[veronica]

Also check values of maximal_backoff_time and queue_run_delay.

postconf -n | grep maximal_backoff_time
postconf -n | grep queue_run_delay

Hope you have not changed the default values

Would be easy to track the issue of you could paste the log snippet for more analysis.

[tiger2000]

Thanks for the advices and after several testing, I found something which I hope to share with someone who thought helpful.

when I first time to send an email to 16000 recipients , it almost took 30 minutes when i saw the last record finished in /opt/zimbra/log/mailbox.log.

and after 5 or 10 minutes, I sent another email again to such 16000 recipients. it just took about 12~15 minutes.

Therefour, I suppose that I should always reboot the system just to meet the real case.

Ok, so that it now always takes 25~30 minutes in every test.

Well, keep post short, and here's my result:
I found this two parameters are very useful to my case, and they are :

1. disable lucene write cache by :
zmlocalconfig -e zimbra_index_max_uncommitted_operations=0

2. set batchindexsize to 20 by :
zmprov mc default zimbraBatchedIndexingSize 20

it redunces the delivery time from 30 minutes to 10~12 minutes.
and please take care about the "zimbraBatchIndexSize" , if you increate this value too large, sometimes it will cause your "mta" failed to start.


by the way, i've also changed these mysql parameters:
1. table_cache
2. innodb_open_files
3. innodb_buffer_pool_size (check your RAM size first, before changing this value)
However, i don't find any significant improvement, thus I think Lucene is the bottleneck in my case.

my testing environment is on zcs 5.0.13 (running on P4 dual-core, 4G RAM, 200SATA hd, centOS 4.4 32-bit), and anti-spam and anti-virus are both disabled. and running "iostat" during email delivery shows regular iowait is in 40%~80% range, and CPU idle percentage is about 30%~5% (looks like I have CPU bottleneck here ??)

any other comment will be highly appreciated.
Thanks.