Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top
root@host [/home4/nudenude/public_html/hello/cache]# top -r
        top: unknown argument 'r'
usage:  top -hv | -abcHimMsS -d delay -n iterations [-u user | -U user] -p pid [,pid ...]

root@host [/home4/nudenude/public_html/hello/cache]# top
top - 09:15:22 up 32 days,  7:12,  2 users,  load average: 274.18, 268.12, 262.50
Tasks: 863 total, 215 running, 630 sleeping,   3 stopped,  15 zombie
Cpu(s): 13.8%us, 85.7%sy,  0.4%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.2%si,  0.0%st
Mem:  16313868k total,  7121660k used,  9192208k free,   679320k buffers
Swap:  4194296k total,   311896k used,  3882400k free,   478132k cached

 6051 nudenude  20   0  103m  29m 4892 R  4.2  0.2   0:14.04 php
 6472 nudenude  20   0 97048  26m 5272 R  4.2  0.2   0:13.07 php
 7181 nudenude  20   0  100m  30m 4928 R  4.2  0.2   0:11.31 php
 7196 nudenude  20   0  105m  26m 5264 R  4.2  0.2   0:11.19 php
 8003 nudenude  20   0  105m  32m 5288 R  4.2  0.2   0:09.42 php
 8008 nudenude  20   0  107m  29m 4960 R  4.2  0.2   0:09.42 php
 8050 nudenude  20   0  105m  34m 5276 R  4.2  0.2   0:09.32 php
 8052 nudenude  20   0 98664  23m 5300 R  4.2  0.1   0:09.32 php
 8067 nudenude  20   0  104m  28m 5296 R  4.2  0.2   0:09.31 php
 8257 nudenude  20   0 96608  25m 5304 R  4.2  0.2   0:09.02 php
10640 nudenude  20   0 96032  22m 4932 R  4.2  0.1   0:04.40 php
12082 investgr  20   0 87588  16m 4928 R  4.2  0.1   0:00.23 php
 5962 nudenude  20   0  103m  33m 4900 R  3.9  0.2   0:14.19 php
 6095 nudenude  20   0  105m  29m 5276 R  3.9  0.2   0:13.96 php
 6467 nudenude  20   0  103m  28m 5280 R  3.9  0.2   0:12.97 php
 6488 nudenude  20   0  102m  26m 5280 R  3.9  0.2   0:12.96 php

My server got 16 GB memory. Of which 7GB is still free. Yet the server uses 4GB SWAP memory and 85.7% of memory is used by system.

Why? Also look at the colum. VIRT 103m, RES 29m. So most memory is not even physical. Why?

share|improve this question
311896k is not 4G, its 312M. Often the Linux kernel decides that some memory areas seem to be almost not used, so it prefers to move them to the slower disk, instead of keeping them in the faster Memory. You can influence this behaviour using the swappiness setting in /proc/sys/vm/swappiness – mauro.stettler Feb 16 '13 at 16:37
See also en.wikipedia.org/wiki/Swappiness – ott-- Feb 16 '13 at 16:51
Your 85.7%sy is CPU usage by the system (i.e., in-kernel, running system calls, others). That number is very high, something is using the system too much (complex firewall rules? just a wild guess). – vonbrand Feb 16 '13 at 16:52
How to find that out? – Jim Thio Feb 16 '13 at 17:04
7GB of RAM weights less than 250 grams, so there is not "tons of physical memory unused". – BatchyX Feb 16 '13 at 19:06
up vote 5 down vote accepted

"VIRT 103m, RES 29m. So most memory is not even physical. Why?"

Since it's sort of my fault for turning you on to this difference I'll try and explain further.

If you've done some programming, you may be aware of what a memory address is -- it's (generally) a 32 or 64 bit number (hence, 32 and 64 bit computers) that the system uses to organize bytes of memory; every byte of memory has an address. If you are not familiar with this, that's all you need to know right now.

The addresses you encounter in programming on a modern OS are not "real" in the sense of being unique and referring to a specific unique place in RAM. In other words, address 0xdeadbeef (that's a 32 bit number in hexadecimal, which is usually how addresses are represented) in one program is not the same place as 0xdeadbeef in some other program. Once upon a time they were; in other words, the system had just one set of addresses, starting at 0x000000, and this address space was shared between everything, including the kernel.

There's a number of reasons why this is not the case anymore, one of which is security. Different programs are not suppose to be able to access one another's memory, so there is no reason they need addresses for it. Anyway, what happens now instead is the kernel provides each process with a virtual address space (see also my answer to your other answer) starting at 0x0000000. This is organized so that parts of it map onto other entities such as shared libraries, etc. The process's own private space is divided into various segments, and most of it is purely theoretical -- it's technically part of the "heap" section, but programs rarely use the entire heap available to them.

The VIRT score in top is taken from numbers the kernel reports via the /proc interface. It's all the addresses in all the parts that have actually been committed to something specific. In other words, it is the amount of the program's virtual address space available that has actually been made use of.

That's still not the same thing as real memory. Saying you might use something in a specific way is not the same thing as using it. For example, if I declare an array of 100 MB but put nothing in the array, I just added 100MB to my VIRT score, but nothing (or nearly nothing, you need one byte to hold the address of the array) to the RSS/RES ("RESident Set Size") score. The kernel is quite clever with memory management this way. Maintaining a table that maps virtual addresses to real address means that the list of virtual address can be much larger than the corresponding list of real addresses, because a lot of the virtual addresses don't correspond to anything because the program hasn't actually tried to access them -- it just asked for them to be created.

When the program goes to access an address that hasn't been mapped yet, the kernel provides some real memory to do so. Hence, the amount of "virtual address space" consumed will always be greater than the amount of "real memory" in use. Usually it's a lot bigger, but the specific reasons for that are beyond the scope of this explanation ;)

You could maybe think of VIRT as a line of credit you've arranged and RES as your actual debt. This analogy is complicated by the fact that RES includes parts which may be shared with other programs (generally, common libraries); those parts are real but their score is duplicated in all processes which access them. The linux kernel reports a "Pss" score which is, eg:

  • 100% of private space
  • 25% of shared space A, where 3 other processes also use A.
  • 50% of shared space B, where 1 other process uses B.

When one of the other processes using A ends, your Pss will go up accordingly (to 33%). Top does not report this figure, however.

Why linux uses swap memory when there are still tons of physical memory unused

It has to do with two priorities that place some importance on leaving some physical memory free. When stuff is swapped out under those circumstances, it belongs to processes that are still loaded but have been idle for a long time. If/when they get used again, they will be loaded from swap back into physical memory (swapping something else out if necessary).

The first reason for wanting to keep a pool of free memory has to do with with the stuff discussed above; while the bank (kernel) has probably issued way more lines of credit WRT memory than it could fulfill all at once, chances are it won't have to. However, it probably will have to fulfill some part of them in the very near future. So the idea is that active processes can be made more responsive by keeping a chunk of change at hand; if long idle processes were left in RAM until it was completely full, swapping would become more noticable and slow the most active processes down, because it wouldn't happen until a critical point.

The other reason is that the free RAM is actually used as a file cache by the kernel (this is why the first line of free output is usually very different than the second line). The purpose of the file cache is to keep things in memory that are frequently used, but not actually used right now, in case they are needed again in the immediate future. When a running process is done with something, it is marked as free memory but left there as part of the file cache. If that or another process then asks for that thing again, it is just marked as part of that process (no longer free), meaning it doesn't have to be loaded from disk again. This also provides responsiveness and increased performance to active processes.

You can adjust how eager the kernel is to pursue this policy by adjusting the /proc/sys/vm/swappiness score. However, unless you have some particular reason to do so, you should leave it as is -- a lot of thinking and experience has gone into determining how to keep the system most responsive under a range of circumstances.

share|improve this answer
Excellent answer, I'd just add (for clarity) that this "shared" memory includes all dynamically linked libraries (kernel keeps them loaded once in physical memory, but each program that links to it, also gets it in its virtual memory). – orion Jan 14 '15 at 11:18

mmap'ed files will also show up as swap [read swapable or swaped].

swapon -s will tell you whether the swapped space comes from your configured paging devices or not. If not it is a safe bet that your processes attach to files on disk.

I think program executables swap out this way automatically - since the program is already on disk it can be loaded from there again if it needs to be swapped out. Not sure what happens if an executable is overwritten while a program is running.

VIRT includes pages mapped in from files, swap space used, and space that has not been physically allocated (yet). RES is actual RAM consumed, but parts of it overlaps shared libs (eg you only get back the full amount shown under RES when the last process that shares each shared object exists.

paxdiablo wrote a good answer to a related question here: http://stackoverflow.com/questions/1972765/mmap-problem-allocates-huge-amounts-of-memory


share|improve this answer
Interesting pointer +1 for that. – Nils Feb 17 '13 at 21:26

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.