Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

Every Saturday, for reasons unknown to me, one of my servers experiences a spike in swap utilisation. It's not necessarily a problem as there is plenty of free memory available, but I'd still like to understand what's going on.

In particular, I'm perplexed by a situation where swap goes from 2% used to 100% used over a period of around 20 minutes. When this happens though, there doesn't seem to be any increase in the swap rate. The server has an 8 GB swap partition, so I would expect a page out rate of thousands of pages a second in order to fill the available space.

Furthermore, there doesn't seem to be any spike in memory utilisation at the time that causes the server to swap.

Can anyone give an explanation for the observed behaviour? The server is RHEL 4.8 (old, I know) with a 2.6.9 Kernel. I've included some output from sar below.

Swap utilisation jumps from 2% to 100% in around 20 minutes:

$ sar -r -f sa12
....
06:00:01 PM kbmemfree kbmemused  %memused kbbuffers  kbcached kbswpfree kbswpused  %swpused  kbswpcad
06:10:01 PM   4583856  11847032     72.10     79676    319804   8178056    207864      2.48     19816
06:20:01 PM   4720904  11709984     71.27     38840    225108   8178400    207520      2.47     19124
06:30:02 PM   4839160  11591728     70.55      1404    144948   1968152   6417768     76.53   6227068
06:40:02 PM   4827016  11603872     70.62      1640    145484     10232   8375688     99.88   8182968
06:50:02 PM   4836376  11594512     70.57       844    123304        60   8385860    100.00   8193044
07:00:01 PM   4825764  11605124     70.63       920    128108      4516   8381404     99.95   8188680
....

Here are the swapping statistics for the same period.

$ sar -W -f sa12
....
06:00:01 PM  pswpin/s pswpout/s
06:10:01 PM      0.00      0.00
06:20:01 PM      0.00      0.00
06:30:02 PM      0.68      0.00
06:40:02 PM      1.85      0.00
06:50:02 PM      5.07      0.00
07:00:01 PM      8.62      0.00
....
share|improve this question
    
What runs from cron at saturdays? –  ninjalj Oct 14 '13 at 13:13
    
Nothing I am aware of, though I'm yet to log on and catch the server while it's swapping. The server runs an Adobe CQ CMS, so it may be some internal process of that system. That said, whether it's CQ or cron, I don't understand what could cause swap utilisation to spike without causing the swapout numbers to rise. –  Vortura Oct 14 '13 at 15:26
    
What is the value of /proc/sys/vm/swappiness? –  jordanm Oct 16 '13 at 4:45
    
It's the default, 60. Therefore, I absolutely expect this server to swap idle processes in order to keep memory available for the page cache. However, I don't see any obvious memory pressure at the time the swapping occurs, and it still doesn't explain the ~0 swap out rate. –  Vortura Oct 16 '13 at 7:24
add comment

1 Answer

This doesn't directly answer your question, but the following script may be useful in your investigation. It lets you know how much swap space each individual process is using:

Find out what processes are using swap

If you have a rough idea of when the swapping starts, you could set cron to run this script around that time.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.