Linux file cache, why so small?

[Andrewb]

Clean CentOS 4.3, Zimbra OS

top shows the following memory parameters:
Mem: 1034712k total, 402588k used, 632124k free, 672k buffers
Swap: 2096376k total, 587468k used, 1508908k free, 35856k cached

Looks like file cache is not used or used poorly for zimbra operations? Is this something specific to java?
Can someone look at their values?

This server (P4 processor) looks to be using HDD pretty much, at least HDD lamp blinks a lot during email operations while we have only 20 users running.

I thought linux would like to use as much memory for cache as it can, but it just leaves 632124 free and cache stays at this value all the time. Is this normal operation?

[KevinH]

Email in general is hard on disks. Lots of reads and writes. We don't do anything to disable filesystem caching and also cache data in our server. What's the CPU load like on the system? How's the performance? Are you just poking around or do you see a real end user performance problem?

[Andrewb]

Processor is working at 2% most of the time but with 100% peaks on some of the user requests.
What is strange, and I beleive it is not really a zimbra issue, is that there is 587468K in swap while 632124 free. I tried turning off the swap and the system looks to be working faster on some actions like moving a lot of old email into Zimbra through IMAP account.

But it is not correct to have swap off as the server may fail reaching the memory limit.

Mem: 1034712k total, 402588k used, 632124k free, 672k buffers
Swap: 2096376k total, 587468k used, 1508908k free, 35856k cached

[KevinH]

Also remember if your pushing new mail in via IMAP each message needs to be indexed in addition of being written to disk. These migration/import activities shouldn't be considered normal usage and you should expect a sluggish system if your pounding the disk(s) with that activity.

BTW: Do you have local disks only or is there a multi-disk RAID setup?

[Andrewb]

Originally Posted by KevinH

Also remember if your pushing new mail in via IMAP each message needs to be indexed in addition of being written to disk.

This is correct. I think it is mostly a linux issue of how it handles swap operations. Some of our users could run a large IMAP push and the memory consumption could grow. After the operation finished the memory was cleared but some of data still remained in the swap area. I will be watching these parameters more and will post other thought on this later.

Originally Posted by KevinH

BTW: Do you have local disks only or is there a multi-disk RAID setup?

We have software mirrow.