Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

Our Linux server is responding very slowly. top doesn't show any excessive CPU usage. I noticed that even though there's about 5GB of free memory, the system is using all of swap with no free swap left. Could this be reason for system running slow? Is there any solution other than reducing the number of processes?

Secondly, even though there is free memory available, why is Linux swapping already? I thought swap is used only when there’s not actual memory left.

free -m
             total       used       free     shared    buffers     cached
Mem:         32045      26218       5826          0        127        123
-/+ buffers/cache:      25967       6077
Swap:        16387      16387          0

Update:

  • swapiness is at default level: 60
  • I don't think it's a numa system
  • I see a couple of java processes running with 8GB heap: -Xms8000m -Xmx8000m

Seems nuts on first look, but possibly someone had reasons for doing this. I think this is what's eating up most of swap, but I need to further investigate how the java heap affects Linux swap and memory/performance in general. Any pointers on impact of above java heap config on system performance will be really helpful.

share|improve this question
    
Is this a numa system? Maybe the resources are not distributed evenly (e.g. one process taking a lot of memory) and one node has no memory left? –  Ulrich Dangel Jun 25 '12 at 4:33
3  
Using swap is not necessarily bad. Use vmstat or iotop to verify that there actually is swap activity. –  mgorven Jun 25 '12 at 5:27
    
If you have some disk space to spare, you could add a swap file and see a) if the performance goes back up as long as there is still swap left and b) how long it takes until the swap is used up again. –  Christian Jun 25 '12 at 7:19
1  
Running with no swap left is likely to slow a system down, as it no longer has enough RAM left to comfortably keep disk files in cache. –  Gilles Jun 26 '12 at 1:02
add comment

2 Answers

How many Java processes have been initialized with the 8GB heap? Perhaps keep the maximum at 8 and lower the initial heap size. If you have the disk space, experiment with increasing the swap size as well

share|improve this answer
add comment

Do you have tmpfs mounts? Those are backed up by swap and may be the cause for zero swap space available. Do syslog or dmesg show anything unexpected? Are you sure that your system is not constantly swapping? Check vmstat -SK 1 and verify columns si and so (swapped pages in and out in KB/s). If you need to know which processes are swapping, run sudo iotop -od5. If you're running Ubuntu, you need delayacct kernel flag for full statistics. For me, your system seems to have really small cache and buffers for such a big system memory.

The swap is used to improve whole system performance. Example of this would be to swap out parts of a big program that's currently sleeping to allow using more memory for disk cache or file buffers. If you were to disable the swap in this case, system would effectively have smaller disk cache and file buffers. Because your swap space is full, you may effectively suffer from the same issue.

The swap space is especially important if you have any programs that "leak" memory (that is, a program acquires blocks of memory that are not really used). Such a "leak" may not be an error in the program, it may just happen that the memory is so seldom used that it may be better treated as leaked. Without swap space, such leaked memory cannot be used as cache or buffers.

A correctly running OS with virtual memory is using swap pretty much always unless the whole working set (all the accessed files, all memory reserved by applications and all write buffers) fits in the memory fully for the entire runtime of the system. In most cases this is only true if the system is constantly rebooted or system has huge amount of RAM compared to whole filesystem size.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.