I've been using the Outlook connector in a demo install of Zimbra Professional edition. Since it's only been a few months, I don't have as much mail or subfolders as your users, but I've seen quite good performance.
Note: my ZCS is Release 5.0.12_GA_2789.MACOSXx86 Network Edition. Outlook connector is the same version
Client is on XP SP2 (maybe SP3 by now) and Outlook is 2007 with the kb961752 hotfix installed.
While I haven't personally had problems, you should be aware that others have reported a bug when using this hotfix, or Office 2007 SP2. See
Bug 37146 - Sender shows as "unknown" after applying Outlook 2007 SP2 Hotfix Sent items are from "unknown" Outlook 2007 Service Pack 2
Some indications are that the problem only occurs with certain versions of ZCO.