I have two servers, server1 and server2. Both of them are identical HP blades, running the exact same OS (RHEL 5.5). Here's the output of free for both of them:
### server1:
total used free shared buffers cached
Mem: 8017848 2746596 5271252 0 212772 1768800
-/+ buffers/cache: 765024 7252824
Swap: 14188536 0 14188536
### server2:
total used free shared buffers cached
Mem: 8017848 4494836 3523012 0 212724 3136568
-/+ buffers/cache: 1145544 6872304
Swap: 14188536 0 14188536
If I understand correctly, server2 is using significantly more memory for disk I/O caching, which still counts as memory used.
But both are running the same OS and if I remember correctly, I configured both with the same parameters when they were installed. I did a diff on /etc/sysctl.conf and they are identical.
The problem is, I am collecting memory usage and other metrics over a period of time, (eg: vmstat, iostat, etc.) while a load is generated on the system. The memory used for caching is throwing off my calculations on the results.
How can I measure actual memory usage from my running processes, rather than system usage? Is used - (buffers + cached) a valid way to measure this?