Understanding and configuring cache memory

Question

I have a few questions about cache memory.

On my system "free -h" command gave me the below output,

      total        used        free      shared  buff/cache   available
Mem:   7.6G        2.1G        1.5G         46M        4.0G        4.8G
Swap:  1.6G         28M        1.6G

1. Why is "cache memory" required inside the main memory?

   As far as I know "cache memory" is different from the main memory (RAM). It is very costly and it is much faster. Please correct me if I am wrong.

2. Before 1 hour this cache memory was 3 GB. Why has it now increased by 1 GB?

   Please note that I didn’t start/run any new processes.

3. Till what limit can I start new processes/applications?

   I mean as per the free -h command output,

   free memory = 1.5 GB, Cache memory = 4 GB and Swap memory = 1.6 GB
   

   Can I start a new application up to (1.5 + 1.6) GB or (1.5 + 1.6 + 4) GB?

4. Can I set/configure this cache memory value?

   If yes, then how?

Answer 1

1. I shall correct you! The expensive thing is CPU cache.
2. Since this is disk cache, it's used by any file access.
3. Bad news: this is complicated and nasty.
4. Good news: you shouldn't need to worry about it too much nowadays. 8 GB of RAM is plenty for a general-purpose desktop system.

1. I shall correct you! The expensive thing is CPU cache.

The CPU has a small bank of fast internal RAM. Data from main memory which is frequently accessed is copied to this cache, automatically by the CPU.

As explained elsewhere, free shows disk cache. It does not show the CPU cache. The disk cache does the same thing, except for disk blocks. It's stored in main memory, and it's managed by the operating system.

So we have three different tiers of memory! This is described as a "memory hierarchy". The main memory we use for the disk cache actually is faster and more expensive per byte - if we compare it to the disk.

2. Since this is disk cache, it's used by any file access.

So it can be used without loading any new programs, just by using them. For example, when you visit websites they are copied to disk (yet another caching strategy!)

When I said "frequently accessed" data is cached, I lied. It's simplest and most efficient to cache everything...

The disk cache will grow until you have no free memory left...

When you need more memory, some blocks will be evicted from the disk cache to make room. This eviction is controlled by a policy designed to maximize efficiency. E.g., "LRU": evict the least recently used block.

3. Bad news: this is complicated and nasty.

Windows was developed with some concern for these issues; if you tried to open too many applications, it's supposed to stop you and complain, which lets you learn roughly how much you can keep open at once.

On Linux, the expected behavior is that you start filling up swap and the system suddenly becomes too slow to recover. Or if you don't have swap, a similar thing could happen where the cache of program code and other such essential files are evicted. If you somehow manage to make enough progress despite this, to run out of memory for pages which aren't backed by the disk, the Out Of Memory killer will start murdering processes it doesn't like to reclaim memory.

You can still learn roughly how much is too much. You just end up having to hard-reboot in order to recover :).

You can configure it to prevent "overcommit", and keep swap disabled or very small. That's the closest you can get to Windows. However, a lot of Linux code is designed with the assumption of "overcommit". If you've got plenty of RAM it's not too much of a problem. And the disk cache will still act as a less dangerous sort of overcommit; you don't necessarily end up needing a lot of RAM that you never get any advantage from.

In practice... some people prefer to run with no/very small swap, which is fine. But it's not really common to disable overcommit. So it's not a recommendation, and if you do enable it then you're going to be in a different world. E.g., if you ask about a problem and it's actually related to disabling overcommit, not many people will have the experience to recognise it.

4. Good news: you shouldn't need to worry about it too much nowadays. 8 GB of RAM is plenty for a general-purpose desktop system.

4 GB works equally well right now and is not expensive. It's just a constraint if you want to run more than one system, i.e., a virtual machine. I tend to run with 4 GB nowadays. I can't remember the last time I had a problem running out of memory. My experience with this is from older systems with much less memory.

The sysctls mentioned by another answer do not limit disk cache. They limit how much disk cache you can have which is "dirty". This refers to cache blocks which a program has written data to, which have not yet been synced back to the disk. This type of caching is described as a "write-back cache".

---

That said, to get a rough idea of a % utilization, I would

1. Account for the total of "used" and "buff/cache" in your calculation for a comprehensive view.
2. Measure on a fresh boot to exclude old cached files. Or a fresh login after running sync; echo 3 | sudo tee /proc/sys/vm/drop_caches. I expect the latter gives a slightly lower result. It's arguable which of these is more realistic. Obviously dropping caches is a more artificial test, but if the dropped files are actually needed for your workload they'll get read back in anyway.
3. This is on the assumption that you're not ending up with massive data files in disk cache, when you wouldn't mind if they didn't fit and would be happy with mere disk performance for them. The sort of thing that could happen if you measured after playing through a video file. In this case it will be harder. If the full video file fitted in memory, you could just subtract its size from buff/cache.

You can verify that X amount of memory is sufficient for your workload, e.g., by booting the kernel with the option mem=X. Like mem=256M if you want to test 32-bit software on the same amount of RAM as the original Raspberry Pi model A.

Answer 2

1. Why is "cache memory" required inside the main memory?

I think when referring to "cache memory" you're thinking of memory used for disk caching, which is RAM used to store information from a hard disk to speed up future processes. It will be freed when future processes have to use information that is not already present in memory and then be used by those new processes.

2. Before 1 hour this cache memory was 3 GB. Why has it now increased by 1 GB?

This is nothing bad and/or unusual, as unused memory is wasted memory. The system probably preloads code it deems usable in the future.

3. Till what limit can I start new processes/application?

Basically until the available memory after buffers/cached is zero. When this happens, your system will start using swap memory on disk (note this is not RAM, but space on your hard disk). This is undesirable as disk speeds are considerably lower than RAM speeds and will result in your system becoming really slow.

4. Can I set/configure this cache memory value?

It's Linux/Unix, so of course it is configurable :) Here's a guide on configuring Linux disk caching.

See also this answer which gives a good explanation on each field. And this for a more humorous approach: www.linuxatemyram.com.

Answer 3

Cache is one of the biggest performance benefits of the Linux operating systems.

1. Why "cache memory" is required inside the main memory?

This is how basically Linux OS is being designed from the start. The Linux kernel will use the available memory for disk caching, unless some of it is required by some application. This slightly improves performance. When there are no more physical memory pages available, the kernel swaps some older pages back to disk

2. Before 1 hour this cache memory was 3 GB. Why has it now increased by 1 GB?

If physical memory is not in use, the kernel will put it to work as a disk buffer cache. The disk buffer stores recently accessed disk data in memory. If the same data is needed again it can be quickly retrieved from the cache, improving performance.

3. Till what limit can I start new processes/application?

You can start your application whenever you want OS will decide how much physical memory will be allocated, but this has nothing with cache. Normally when running some abnormally high extensive application you will most likely end with a Linux Out Of Memory (OOM) kernel problem, but this is more related to the application itself and how application is designed and coded rather than OS settings.

4. Can I set/configure this cache memory value?

You can check on this link Clear cache/buffers how to clear cache/buffers. As far as I know, there isn't any possibility to set static values for the cache.