We have a server that at least once a week will suffer from very sluggish performance. Memory will go into swap by about 10-15GB and the server has 36GB of RAM.
My question is can i find either when the server went into swap or what caused it to go into swap.
I have a feeling there is a single processes that is causing the issue but when i check the process list it all seems normal. The server is already into Swap at this point though.
This seems like a good article to detail about swapiness:
https://en.wikipedia.org/wiki/Swappiness
But I wonder if poor performance is due to swapping or "something else".
Swapping because programs have a "working set" which changes over time, and the idea is to swapout the "unneeded pages".
A system "starts swapping" when the existing ram isn't large enough.
Slugishness may indicate the run queue is getting too high (the number of processes which want to run) -- if the memory load is a function of users/programs, more users will want to run more programs (and it may be coincidental swapping occurs when the system is slow).
This also looks pretty good:
http://www.tecmint.com/commands-to-monitor-swap-space-usage-in-linux/
(I never knew about the glances command -- always learning something new)
These should point you at tools to help diagnose your problem.
