Hot answers tagged

37

Clean pages are pages that have not been modified since they were mapped (typically, text sections from shared libraries are only read from disk (when necessary), never modified, so they'll be in shared, clean pages). Dirty pages are pages that are not clean (i.e. have been modified). Private pages are available only to that process, shared pages are mapped ...


33

What might be happening if a process is "killed due to low RAM"? It's sometimes said that linux by default never denies requests for more memory from application code -- e.g. malloc().1 This is not in fact true; the default uses a heuristic whereby Obvious overcommits of address space are refused. Used for a typical system. It ...


31

It isn't sticky - you just write to the file to make it drop the caches and then it immediately starts caching again. Basically when you write to that file you aren't really changing a setting, you are issuing a command to the kernel. The kernel acts on that command (by dropping the caches) then carries on as before.


21

It sounds like you've got a decent grasp on what happened. Yes, because you hard-powered-off the system before your changes were committed to disk, they were there when you booted back up. The system caches all writes before flushing them out to disk. There are several options which control this behavior, all located at /proc/sys/vm/dirty_* [kernel doc]. ...


14

This won't happen to you if you only ever load 1G of data into memory. What if you load much much more? For example, I often work with huge files containing millions of probabilities which need to be loaded into R. This takes about 16GB of RAM. Running the above process on my laptop will cause it to start swapping like crazy as soon as my 8GB of RAM have ...


10

The text segment is the mapping at 0x400000 - it's marked 'r-x' for readable and executable. The mapping at 0x600000 is read-only, so that's almost certainly the ".rodata" section of the executable file. GCC puts C string literals into a read-only section. The mapping at 0x601000 is 'rw-', so that's probably the famed heap. You could have your executable ...


8

It appears that the stack memory limit is not allocated (anyway, it couldn't with unlimited stack). https://www.kernel.org/doc/Documentation/vm/overcommit-accounting says: The C language stack growth does an implicit mremap. If you want absolute guarantees and run close to the edge you MUST mmap your stack for the largest size you think you will ...


8

When swap starts to be used is dependent on how you have your swappiness kernel parameter set. At swappiness 0 swapping will only occur when memory is fully used and at 100 it will occur as soon as possible. The default value is 60. OOM errors occur when RAM and swap are completely full.


7

Both. A page is the smallest unit of memory that the hardware page mapping function of the CPU deals with. If for example, pages are 4096 bytes in size, then each page begins and ends on a boundary aligned to 4096 bytes in both physical and virtual memory and all 4096 bytes in the page are mapped linearly and contiguously from virtual address to physical ...


7

Well. How's that for fried RAM? Guess that was the culprit, after all. I'm pleased to report that, after removing the defective stick, everything is going quite smoothly.


6

Like the docs say, user space gets 247 bytes = 128TiB, and kernel gets 512MiB. The rest of the address space goes to various parts of the system, along with a few unusable holes.


6

Edit: This answer is incorrect. Though still a possible cause for oom-killer to be invoked, it is not the cause in this specific case. It looks like this is due to memory fragmentation. From the output you provided, the highest order contiguous memory block you have is a 32kb block in the normal zone. This means that if anything tries to allocate a chunk ...


6

From the kernel documentation, in Documentation/x86/x86_64/mm.txt: Virtual memory map with 4 level page tables: 0000000000000000 - 00007fffffffffff (=47 bits) user space, different per mm 247 bytes = 128TiB


5

Linux as well as Windows, work pretty much the same here. Every process gets it's own "virtual" address space. This doesn't mean that the memory is actually physically available (obviously most 32bit computers never had enough memory), that's, why it's virtual. Also the addresses used there don't correspond to the physical addresses. Thereby physical memory ...


5

swapon have -p switch which sets the priority. I can set up: swapon -p 32767 /dev/zram0 swapon -p 0 /dev/my-lvm-volume/swap Or in /etc/fstab: /dev/zram0 none swap sw,pri=32767 0 0 /dev/my-lvm-volume/swap none swap sw,pri=0 0 0 EDIT: Just for a full solution - such line may be helpful as udev rule: KERNEL=="zram0", ACTION=="add", ...


5

Sidenote: because of per-cpu locking, it is important to have as many zram-swaps as CPUs (modprobe zram_num_devices=n zram) instead of a single big one. RTFM!


5

As far as I understand your question it happens usually in such way: If you allocate memory: Mark memory as allocated but don't allocate physical memory (hence on access there will be page fault). In Linux it stops at this stage but it is possible that system may allocate physical space immediately - then it performs similar algorithm at the end as on ...


5

32-bit processes can only allocate up to 1, 2, 3, or about 4GB, depending on which memory split was chosen when the 32-bit kernel was built. 32-bit processes on a 64-bit kernel can allocate about 4GB. 64-bit processes on a 64-bit x86-64 kernel can allocate up to 128TiB.


5

Huge pages are for allocating chunks of memory, not writing them. Normally when applications need large amounts of memory, they have to allocate many "pages". A page is simply a chunk of physical memory. Normally this chunk is only a few KB. So when an application is doing a lot of memory intensive operations that span many pages, it's expensive for the ...


5

There is more than one definition of the chunk size for memory writes. You could consider it to be: the width of the store instruction (store byte, store word, …), typically 1, 2, 4, 8 or 16; the width of a cache line, typically something like 16 or 64 bytes (and different cache levels may have different line widths); the width of the memory bus, which is ...


5

The tool cpuid can make a call into the CPU to get more detailed information about the CPU's architecture: TLB size, entires, and associativity $ cpuid | grep -i tlb cache and TLB information (2): 0x5a: data TLB: 2M/4M pages, 4-way, 32 entries 0x03: data TLB: 4K pages, 4-way, 64 entries 0x55: instruction TLB: 2M/4M pages, fully, 7 ...


5

The best way I can attempt to answer those questions is to say what those three actually are. zRAM zRAM is nothing more than a swap device in essence. The memory management will push pages out to the swap device and zRAM will compress that data, allocating memory as needed. Zswap Zswap is a compressed swap space that is allocated internally by the kernel ...


5

The run-time library calls sbrk to read the current end of the data segment. sbrk is implemented on top of the brk syscall; as you can see, calling brk with an invalid address just returns the current value.


5

There is no command that gives the “actual memory usage of a process” because there is no such thing as the actual memory usage of a process. Each memory page of a process could be (among other distinctions): Transient storage used by that process alone. Shared with other processes using a variety of mechanisms. Backed up by a disk file. In physical ...


5

It may be a huge doc to start, but I think it's worth the time you'll need to read it : Have look on the "Linux-Insides" doc, more precisely the Memory Management part. You can also read it through Gitbooks Have fun.


5

The "marking that memory as unused" is a function of how much work the unlinkat(2) system call has to do, which in turn scales linearly with the size of the file. For a default tmpfs on a RHEL 6 system with ~4G of memory, this can be demonstrated as follows. $ sudo mkdir /tmpfs; sudo mount -t tmpfs -o size=75% tmpfs /tmpfs; cd /tmpfs $ dd if=/dev/zero bs=1M ...


4

Physical Address Extension (PAE) sounds exactly like what he's referring to. A 32-bit CPU can only map ~4gb of memory, even if the system has more. But with PAE, you can use >4gb, though only 4gb of it is mapped at any one time (a single process will never be able to use >4gb). So basically when the kernel changes the actively running process, it re-maps ...


4

Processes aren't killed when there is no more RAM, they are killed when they have been cheated this way: Linux kernel commonly allows processes to allocate (i.e. reserve) an amount of virtual memory which is larger than what is really available (part of RAM + all the swap area) as long as the processes only access a subset of the pages they have reserved, ...


4

A 32-bit process has a 32-bit address space, by definition: “32-bit” means that memory addresses in the process are 32 bits wide, and if you have 232 distinct addresses you can address at most 232 bytes (4GB). A 32-bit Linux kernel can only execute 32-bit processes. Depending on the kernel compilation options, each process can only allocate 1GB, 2GB or 3GB ...


4

The change is reflected immediately. There is no caching along the way. When you read /proc/<pid>/smaps, you actually trigger a traversal of that process's page table. Information about the mappings is accumulated along the way, then displayed, without any caching. The code behind the /proc/<pid>/smaps file is in fs/proc/task_mmu.c, specifically ...



Only top voted, non community-wiki answers of a minimum length are eligible