What is lazy swap reservation?

Question

I am reading the Linux Progamming Interface.

49.9 MAP_NORESERVE and Swap Space Overcommitting

Some applications create large (usually private anonymous) mappings, but use only a small part of the mapped region. For example, certain types of scientific applications allocate a very large array, but operate on only a few widely separated elements of the array (a so-called sparse array).

If the kernel always allocated (or reserved) enough swap space for the whole of such mappings, then a lot of swap space would potentially be wasted. Instead, the kernel can reserve swap space for the pages of a mapping only as they are actually required (i.e., when the application accesses a page). This approach is called lazy swap reservation, and has the advantage that the total virtual memory used by applications can exceed the total size of RAM plus swap space.

To put things another way, lazy swap reservation allows swap space to be overcommitted. This works fine, as long as all processes don’t attempt to access the entire range of their mappings. ...

As far as I know, swap space is a chunk of space in disk, reserved for memory swapping. When those pages in memory are inactive, they are swapped into swap space in disk. It's like a second level cache for memory/ram.

Then what the hell is this lazy swap reservation mechanism?

Let me demostrate my confusion with an example.

1. Some applications create large (usually private anonymous) mappings....

Ok, then assume I malloc a big array 16384(4096*4) bytes (create large (usually private anonymous) mappings), and operate on only a few widely separated elements of the array.

Then some inactive pages are swapped into swap space, right? Let's say 0-4095(4096B), 8192-12287(4096B) are in memory, and all the other inactive pages, 4096-8191(4096B), 12288-16383(4096B) are swapped into swap space.

Then what does it mean by saying:

Instead, the kernel can reserve swap space for the pages of a mapping only as they are actually required (i.e., when the application accesses a page).

Where else can these inactive pages (4096-8191(4096B) and 12288-16383(4096B) ) stay in, if not staying in the swap space? The text seems to indicate that there is a 3rd level cache for swap space.

memory -> swap space (disk) -> ????

Answer 1

Swap isn’t really a second-level cache for memory; it’s one of several backing stores for memory. When the kernel needs to allocate a page of physical memory, but doesn’t have enough free memory, it needs to evict another page; it can only do that if the contents of the evicted page are either discardable, or can be restored from somewhere else. That somewhere else is backing store: it can be a file on disk (e.g. for executables, or mapped files), or some area of swap.

Swap reservation comes into play when memory accounting tracks overcommitting (see table 49-4 in LPI). When overcommitting isn’t allowed, the kernel needs to determine, at allocation time, whether an allocation is possible. For private writable mappings and shared anonymous mappings, this means that it has to have enough address space, and enough room in swap (so that the kernel can guarantee that the contents of the mapped memory can be written there, thus guaranteeing that writes to the mapped memory will never cause SIGSEGV).

Lazy swap reservation is required for overcommit: it means that the kernel can allocate a swap-backed memory map without reserving the corresponding swap space. As mentioned in LPI, this allows programs to allocate much more memory than is really available, and should be requested using MAP_NORESERVE. The reservation then only happens when a page is written to, which means that writes can fail with SIGSEGV or result in the OOM killer stepping in.

This becomes significant for much larger allocations than your 16KiB example. Imagine you want a sparse 64GiB, 262,144×262,144 array, to make your program easier to write: with strict reservation, you’d need to have all that memory available; without strict reservation, you don’t, and only the pages you write to will actually be allocated.

Note that this is all Linux-specific and tightly tied to the chosen system overcommit policy (/proc/sys/vm/overcommit_memory): in modes 1 (always overcommit) and 2 (never overcommit), MAP_NORESERVE doesn’t change anything, it only has an effect in mode 0.

Answer 2

swap space is a chunk of space in disk

A chunk of what kind of space? Virtual Memory.

It is easy for the kernel to give every process as much virtual memory as it wants. That is the lazy, overcommitting part. The example of the sparse scientific array wants to show that it often is a good idea to be lazy. The whole array will not be used at once, so other programs can run at the same time.

It's like a second level cache for memory/ram.

But not really faster than "the" disk, most of the time (without a dedicated swap device)

It is rather the other way around, especially with overcommitting:

Swap turns (physical) RAM into a cache for the storge device(s).

(i.e. RAM is not something to have and hold for a process, but it is the working area for the active pages of every process incl. mapped files).

---

Here it is about overcommitting also swap space, not just RAM. With or without swap it can lead to a situation of OOM after overcommitting.