There will be a couple 'gotchas', so here is my two cents, with some questions, otherwise, it is totally possible:
1.) I would separate out your MX / Antivirus to another box.
2.) Disk IO will be crucial. What types / sizes of disks will you be using?
3.) What type of RAID will be used? Hardware / Software RAID?
4.) RAM, RAM, RAM

I'd recommend going to 8 GB of memory depending on how your users are going to be accessing their e-mail.
Cheers,
DJS