"VIRT 103m, RES 29m. So most memory is not even physical. Why?"
Since it's sort of my fault for turning you on to this difference I'll try and explain further.
If you've done some programming, you may be aware of what a memory address is -- it's (generally) a 32 or 64 bit number (hence, 32 and 64 bit computers) that the system uses to organize bytes of memory; every byte of memory has an address. If you are not familiar with this, that's all you need to know right now.
The addresses you encounter in programming on a modern OS are not "real" in the sense of being unique and referring to a specific unique place in RAM. In other words, address
0xdeadbeef (that's a 32 bit number in hexadecimal, which is usually how addresses are represented) in one program is not the same place as 0xdeadbeef in some other program. Once upon a time they were; in other words, the system had just one set of addresses, starting at 0x000000, and this address space was shared between everything, including the kernel.
There's a number of reasons why this is not the case anymore, one of which is security. Different programs are not suppose to be able to access one another's memory, so there is no reason they need addresses for it. Anyway, what happens now instead is the kernel provides each process with a virtual address space (see also my answer to your other answer) starting at 0x0000000. This is organized so that parts of it map onto other entities such as shared libraries, etc. The process's own private space is divided into various segments, and most of it is purely theoretical -- it's technically part of the "heap" section, but programs rarely use the entire heap available to them.
The VIRT score in top is taken from numbers the kernel reports via the /proc interface. It's all the addresses in all the parts that have actually been committed to something specific. In other words, it is the amount of the program's virtual address space available that has actually been made use of.
That's still not the same thing as real memory. Saying you might use something in a specific way is not the same thing as using it. For example, if I declare an array of 100 MB but put nothing in the array, I just added 100MB to my VIRT score, but nothing (or nearly nothing, you need one byte to hold the address of the array) to the RSS/RES ("RESident Set Size") score. The kernel is quite clever with memory management this way. Maintaining a
table that maps virtual addresses to real address means that the list of virtual address can be much larger than the corresponding list of real addresses, because a lot of the virtual addresses don't correspond to anything because the program hasn't actually tried to access them -- it just asked for them to be created.
When the program goes to access an address that hasn't been mapped yet, the kernel provides some real memory to do so. Hence, the amount of "virtual address space" consumed will always be greater than the amount of "real memory" in use. Usually it's a lot bigger, but the specific reasons for that are beyond the scope of this explanation ;)
You could maybe think of VIRT as a line of credit you've arranged and RES as your actual debt. This analogy is complicated by the fact that RES includes parts which may be shared with other programs (generally, common libraries); those parts are real but their score is duplicated in all processes which access them. The linux kernel reports a "Pss" score which is, eg:
- 100% of private space
- 25% of shared space A, where 3 other processes also use A.
- 50% of shared space B, where 1 other process uses B.
When one of the other processes using A ends, your Pss will go up accordingly (to 33%). Top does not report this figure, however.
Why linux uses swap memory when there are still tons of physical memory unused
It has to do with two priorities that place some importance on leaving some physical memory free. When stuff is swapped out under those circumstances, it belongs to processes that are still loaded but have been idle for a long time. If/when they get used again, they will be loaded from swap back into physical memory (swapping something else out if necessary).
The first reason for wanting to keep a pool of free memory has to do with with the stuff discussed above; while the bank (kernel) has probably issued way more lines of credit WRT memory than it could fulfill all at once, chances are it won't have to. However, it probably will have to fulfill some part of them in the very near future. So the idea is that active processes can be made more responsive by keeping a chunk of change at hand; if long idle processes were left in RAM until it was completely full, swapping would become more noticable and slow the most active processes down, because it wouldn't happen until a critical point.
The other reason is that the free RAM is actually used as a file cache by the kernel (this is why the first line of
free output is usually very different than the second line). The purpose of the file cache is to keep things in memory that are frequently used, but not actually used right now, in case they are needed again in the immediate future. When a running process is done with something, it is marked as free memory but left there as part of the file cache. If that or another process then asks for that thing again, it is just marked as part of that process (no longer free), meaning it doesn't have to be loaded from disk again. This also provides responsiveness and increased performance to active processes.
You can adjust how eager the kernel is to pursue this policy by adjusting the
/proc/sys/vm/swappiness score. However, unless you have some particular reason to do so, you should leave it as is -- a lot of thinking and experience has gone into determining how to keep the system most responsive under a range of circumstances.