Now that you have IDLE in the IMAP server, are you looking to do the same for the web?
The minimum 2-minute lag between email delivery and visibility in the web interface is going to be noticed by our users. Having them bang on "check mail" isn't in anyone's interest.
There are AJAX toolkits out there (used especially in the financial industry; heard some Dow Jones engineers talk about theirs about a year ago) that replace polling with persistent connections and server-push. The DJ guys said they were supporting 50,000 simultaneous real-time push connections per Dell 1850, using custom multithreaded C# code. It "should" be possible to scale Java similarly.