Technically, we should be able to handle 11G, but that's pretty huge. Up until recently, Microsoft recommended a 2G max. They've since changed that, but 11GB is pushing it.
You say Outlook dies most of the time. How does it do that? Do you know how big the mailbox gets before it dies?
There are a few things you can do, of course:
1) Archive things to make your mailbox smaller
2) Download headers only -- you can set that up when you set up the profile. That will make your mailbox a lot smaller
3) If you've got search protocol host -- disable that (or at least disable indexing on zdb's).
4) Disable some virus checking.
If you still have trouble, you can enable ZCO logging (not verbose, just logging and http) so we can see where it dies. The log will be huge, though, and it will perform even more slowly.
If it dies with a crash, then we'll generate a core dump in %TEMP%\zco-cores.
Also, there is a wiki article:
ZCS Connector for Outlook Performance - Zimbra :: Wiki