103

It isn't sticky - you just write to the file to make it drop the caches and then it immediately starts caching again. Basically when you write to that file you aren't really changing a setting, you are issuing a command to the kernel. The kernel acts on that command (by dropping the caches) then carries on as before.


50

Clean pages are pages that have not been modified since they were mapped (typically, text sections from shared libraries are only read from disk (when necessary), never modified, so they'll be in shared, clean pages). Dirty pages are pages that are not clean (i.e. have been modified). Private pages are available only to that process, shared pages are mapped ...


40

What might be happening if a process is "killed due to low RAM"? It's sometimes said that linux by default never denies requests for more memory from application code -- e.g. malloc().1 This is not in fact true; the default uses a heuristic whereby Obvious overcommits of address space are refused. Used for a typical system. It ...


34

Those limits don't come from Debian or from Linux, they come from the hardware. Different architectures (processor and memory bus) have different limitations. On current x86-64 PC processors, the MMU allows 48 bits of virtual address space. That means that the address space is limited to 256TB. With one bit to distinguish kernel addresses from userland ...


22

It sounds like you've got a decent grasp on what happened. Yes, because you hard-powered-off the system before your changes were committed to disk, they were there when you booted back up. The system caches all writes before flushing them out to disk. There are several options which control this behavior, all located at /proc/sys/vm/dirty_* [kernel doc]. ...


21

Any system emulator which emulates a system containing a MMU effectively emulates a MMU in software, so the answer to your question as stated is “yes”. However, virtual memory requires some way of enforcing memory access control, or at least address translation, so it needs either full software emulation of the CPU running the software being controlled, or ...


19

The x86-64 architecture does not use segmentation in long mode (64-bit mode). Four of the segment registers: CS, SS, DS, and ES are forced to 0, and the limit to 2^64. https://en.wikipedia.org/wiki/X86_memory_segmentation#Later_developments It is no longer possible for the OS to limit which ranges of the "linear addresses" are available. Therefore it ...


17

There is no command that gives the “actual memory usage of a process” because there is no such thing as the actual memory usage of a process. Each memory page of a process could be (among other distinctions): Transient storage used by that process alone. Shared with other processes using a variety of mechanisms. Backed up by a disk file. In physical memory ...


16

printf is implemented by the C library, it’s not part of the kernel. (The kernel does have its own equivalent, more or less, but that’s not available to user processes.) So a user process calling printf doesn’t call into the kernel immediately. If printf’s output gets written¹, that happens by calling write, which is handled by the kernel (well, there’s a ...


15

This won't happen to you if you only ever load 1G of data into memory. What if you load much much more? For example, I often work with huge files containing millions of probabilities which need to be loaded into R. This takes about 16GB of RAM. Running the above process on my laptop will cause it to start swapping like crazy as soon as my 8GB of RAM have ...


14

The text segment is the mapping at 0x400000 - it's marked 'r-x' for readable and executable. The mapping at 0x600000 is read-only, so that's almost certainly the ".rodata" section of the executable file. GCC puts C string literals into a read-only section. The mapping at 0x601000 is 'rw-', so that's probably the famed heap. You could have your executable ...


14

It appears that the stack memory limit is not allocated (anyway, it couldn't with unlimited stack). https://www.kernel.org/doc/Documentation/vm/overcommit-accounting says: The C language stack growth does an implicit mremap. If you want absolute guarantees and run close to the edge you MUST mmap your stack for the largest size you think you will need. ...


14

RES is the amount of RAM currently used by the process. This value can vary because memory pages might be swapped in or out. It might even be 0 for a process that has been sleeping for a long time, e.g. an unsolicited daemon. VIRT is the full size of all memory the process is using, whether in RAM or on disk (shared objects, mmaped files, swap area) so VIRT ...


14

As others have said, and as is mentioned in the link you provide in your question, having an 8MiB stack doesn’t hurt anything (apart from consuming address space — on a 64-bit system that won’t matter). Linux has used 8MiB stacks for a very long time; the change was introduced in version 1.3.7 of the kernel, in July 1995. Back then it was presented as ...


13

All of these daemons use WebKit (mostly to show oauth2 login prompts), and WebKit recently introduced gigacages to isolate the heap used by their JS implementation. The allocation for a gigacage is big enough that any access to an arbitrary unsigned 32 bit offset would still land in the gigacage, resulting in these huge allocations. See this blog post for ...


12

Answering things in order: It returns a pointer to the location in virtual memory, and Virtual memory address space is allocated, but the file is not locked in any way unless you explicitly lock it (also note that locking the memory is not the same as locking the region in the file). An efficient implementation of mmap() is actually only possible from a ...


11

No part of the Linux kernel can be paged out, even parts that come from modules. A kernel module can be loaded and (if the module supports it) can be unloaded. This always happens from an explicit request from a userland process with the init_module and delete_module system calls (normally, via the insmod or modprobe utilities for loading, and via rmmod for ...


10

The four records have different permissions, so they can't be merged. The r-xp entry describes a block of executable memory (x permission flag). That's the code. The r--p entry describes a block of memory that is only readable (r permission flag). That's static data (constants). The rw-p entry describes a block of memory that is writable (w permission flag)....


10

No, it’s a bad idea. You shouldn’t think of swap as a mechanism by which you can expand memory; it’s a storage area for parts of memory which don’t have to remain in physical memory, and whose contents don’t exist anywhere else. See Why does Linux need swap space in a VM? for details. If the processes running inside your VMs are running out of memory, you ...


9

Both. A page is the smallest unit of memory that the hardware page mapping function of the CPU deals with. If for example, pages are 4096 bytes in size, then each page begins and ends on a boundary aligned to 4096 bytes in both physical and virtual memory and all 4096 bytes in the page are mapped linearly and contiguously from virtual address to physical ...


9

I don't know why, but I can think of seven reasons why it'd be useful to support twice as much address space as physical memory. The first is so that you can run applications that need the extra memory -- even if it means swapping to disk. Cleaner memory layouts to partition memory usage. E.g., an OS might take higher-numbered addresses and leave lower-...


8

The tool cpuid can make a call into the CPU to get more detailed information about the CPU's architecture: TLB size, entires, and associativity $ cpuid | grep -i tlb cache and TLB information (2): 0x5a: data TLB: 2M/4M pages, 4-way, 32 entries 0x03: data TLB: 4K pages, 4-way, 64 entries 0x55: instruction TLB: 2M/4M pages, fully, 7 ...


8

I would fix it by going back and re-partitioning to give myself some virtual memory, but damn.. You don't need to have a full partition dedicated to swap, and you don't need to re-partition. Create swap as a file is pretty easy. Just create a large empty file, run mkswap on it, then add the swap. # create an big empty 1GB file (or whatever size you like) ...


8

As the x86 has segments, it is not possible to not use them. But both cs (code segment) and ds(data segment) base addresses are set to zero, so the segmentation is not really used. An exception is thread local data, one of the normally unused segment registers points to thread local data. But that is mainly to avoid reserving one of the general purpose ...


8

Memory mapping a file directly avoids copying buffers which happen with read() and write() calls. Calls to read() and write() include a pointer to buffer in process' address space where the data is stored. Kernel has to copy the data to/from those locations. Using mmap() maps the file to process' address space, so the process can address the file directly ...


7

Like the docs say, user space gets 247 bytes = 128TiB, and kernel gets 512MiB. The rest of the address space goes to various parts of the system, along with a few unusable holes.


7

Sidenote: because of per-cpu locking, it is important to have as many zram-swaps as CPUs (modprobe zram_num_devices=n zram) instead of a single big one. RTFM!


7

Linux as well as Windows, work pretty much the same here. Every process gets it's own "virtual" address space. This doesn't mean that the memory is actually physically available (obviously most 32bit computers never had enough memory), that's, why it's virtual. Also the addresses used there don't correspond to the physical addresses. Thereby physical memory ...


7

When swap starts to be used is dependent on how you have your swappiness kernel parameter set. At swappiness 0 swapping will only occur when memory is fully used and at 100 it will occur as soon as possible. The default value is 60. OOM errors occur when RAM and swap are completely full.


7

It depends on exactly what you call virtual memory. An interesting model is the old Win16 model (best known from the old Windows 3.x, not Windows NT). In that model, you had GlobalLock and GlobalUnlock, LocalLock and LocalUnlock functions. These were a form of cooperative, manual management of virtual memory. As this was done in (application) software, it ...


Only top voted, non community-wiki answers of a minimum length are eligible