Hot answers tagged

38

Clean pages are pages that have not been modified since they were mapped (typically, text sections from shared libraries are only read from disk (when necessary), never modified, so they'll be in shared, clean pages). Dirty pages are pages that are not clean (i.e. have been modified). Private pages are available only to that process, shared pages are mapped ...


33

It isn't sticky - you just write to the file to make it drop the caches and then it immediately starts caching again. Basically when you write to that file you aren't really changing a setting, you are issuing a command to the kernel. The kernel acts on that command (by dropping the caches) then carries on as before.


33

What might be happening if a process is "killed due to low RAM"? It's sometimes said that linux by default never denies requests for more memory from application code -- e.g. malloc().1 This is not in fact true; the default uses a heuristic whereby Obvious overcommits of address space are refused. Used for a typical system. It ...


21

It sounds like you've got a decent grasp on what happened. Yes, because you hard-powered-off the system before your changes were committed to disk, they were there when you booted back up. The system caches all writes before flushing them out to disk. There are several options which control this behavior, all located at /proc/sys/vm/dirty_* [kernel doc]. ...


14

This won't happen to you if you only ever load 1G of data into memory. What if you load much much more? For example, I often work with huge files containing millions of probabilities which need to be loaded into R. This takes about 16GB of RAM. Running the above process on my laptop will cause it to start swapping like crazy as soon as my 8GB of RAM have ...


10

The text segment is the mapping at 0x400000 - it's marked 'r-x' for readable and executable. The mapping at 0x600000 is read-only, so that's almost certainly the ".rodata" section of the executable file. GCC puts C string literals into a read-only section. The mapping at 0x601000 is 'rw-', so that's probably the famed heap. You could have your executable ...


9

RES is the amount of RAM currently used by the process. This value can vary because memory pages might be swapped in or out. It might even be 0 for a process that has been sleeping for a long time, e.g. an unsolicited daemon. VIRT is the full size of all memory the process is using, whether in RAM or on disk (shared objects, mmaped files, swap area) so VIRT ...


8

It appears that the stack memory limit is not allocated (anyway, it couldn't with unlimited stack). https://www.kernel.org/doc/Documentation/vm/overcommit-accounting says: The C language stack growth does an implicit mremap. If you want absolute guarantees and run close to the edge you MUST mmap your stack for the largest size you think you will need....


8

When swap starts to be used is dependent on how you have your swappiness kernel parameter set. At swappiness 0 swapping will only occur when memory is fully used and at 100 it will occur as soon as possible. The default value is 60. OOM errors occur when RAM and swap are completely full.


7

Edit: This answer is incorrect. Though still a possible cause for oom-killer to be invoked, it is not the cause in this specific case. It looks like this is due to memory fragmentation. From the output you provided, the highest order contiguous memory block you have is a 32kb block in the normal zone. This means that if anything tries to allocate a chunk ...


7

Well. How's that for fried RAM? Guess that was the culprit, after all. I'm pleased to report that, after removing the defective stick, everything is going quite smoothly.


7

Both. A page is the smallest unit of memory that the hardware page mapping function of the CPU deals with. If for example, pages are 4096 bytes in size, then each page begins and ends on a boundary aligned to 4096 bytes in both physical and virtual memory and all 4096 bytes in the page are mapped linearly and contiguously from virtual address to physical ...


6

From the kernel documentation, in Documentation/x86/x86_64/mm.txt: Virtual memory map with 4 level page tables: 0000000000000000 - 00007fffffffffff (=47 bits) user space, different per mm 247 bytes = 128TiB


6

Like the docs say, user space gets 247 bytes = 128TiB, and kernel gets 512MiB. The rest of the address space goes to various parts of the system, along with a few unusable holes.


6

The tool cpuid can make a call into the CPU to get more detailed information about the CPU's architecture: TLB size, entires, and associativity $ cpuid | grep -i tlb cache and TLB information (2): 0x5a: data TLB: 2M/4M pages, 4-way, 32 entries 0x03: data TLB: 4K pages, 4-way, 64 entries 0x55: instruction TLB: 2M/4M pages, fully, 7 ...


5

Sidenote: because of per-cpu locking, it is important to have as many zram-swaps as CPUs (modprobe zram_num_devices=n zram) instead of a single big one. RTFM!


5

swapon have -p switch which sets the priority. I can set up: swapon -p 32767 /dev/zram0 swapon -p 0 /dev/my-lvm-volume/swap Or in /etc/fstab: /dev/zram0 none swap sw,pri=32767 0 0 /dev/my-lvm-volume/swap none swap sw,pri=0 0 0 EDIT: Just for a full solution - such line may be helpful as udev rule: KERNEL=="zram0", ACTION=="add", ATTR{...


5

As far as I understand your question it happens usually in such way: If you allocate memory: Mark memory as allocated but don't allocate physical memory (hence on access there will be page fault). In Linux it stops at this stage but it is possible that system may allocate physical space immediately - then it performs similar algorithm at the end as on ...


5

32-bit processes can only allocate up to 1, 2, 3, or about 4GB, depending on which memory split was chosen when the 32-bit kernel was built. 32-bit processes on a 64-bit kernel can allocate about 4GB. 64-bit processes on a 64-bit x86-64 kernel can allocate up to 128TiB.


5

Linux as well as Windows, work pretty much the same here. Every process gets it's own "virtual" address space. This doesn't mean that the memory is actually physically available (obviously most 32bit computers never had enough memory), that's, why it's virtual. Also the addresses used there don't correspond to the physical addresses. Thereby physical memory ...


5

The run-time library calls sbrk to read the current end of the data segment. sbrk is implemented on top of the brk syscall; as you can see, calling brk with an invalid address just returns the current value.


5

The best way I can attempt to answer those questions is to say what those three actually are. zRAM zRAM is nothing more than a swap device in essence. The memory management will push pages out to the swap device and zRAM will compress that data, allocating memory as needed. Zswap Zswap is a compressed swap space that is allocated internally by the kernel ...


5

Huge pages are for allocating chunks of memory, not writing them. Normally when applications need large amounts of memory, they have to allocate many "pages". A page is simply a chunk of physical memory. Normally this chunk is only a few KB. So when an application is doing a lot of memory intensive operations that span many pages, it's expensive for the ...


5

There is more than one definition of the chunk size for memory writes. You could consider it to be: the width of the store instruction (store byte, store word, …), typically 1, 2, 4, 8 or 16; the width of a cache line, typically something like 16 or 64 bytes (and different cache levels may have different line widths); the width of the memory bus, which is ...


5

There is no command that gives the “actual memory usage of a process” because there is no such thing as the actual memory usage of a process. Each memory page of a process could be (among other distinctions): Transient storage used by that process alone. Shared with other processes using a variety of mechanisms. Backed up by a disk file. In physical ...


5

It may be a huge doc to start, but I think it's worth the time you'll need to read it : Have look on the "Linux-Insides" doc, more precisely the Memory Management part. You can also read it through Gitbooks Have fun.


5

The "marking that memory as unused" is a function of how much work the unlinkat(2) system call has to do, which in turn scales linearly with the size of the file. For a default tmpfs on a RHEL 6 system with ~4G of memory, this can be demonstrated as follows. $ sudo mkdir /tmpfs; sudo mount -t tmpfs -o size=75% tmpfs /tmpfs; cd /tmpfs $ dd if=/dev/zero bs=1M ...


4

A 32-bit process has a 32-bit address space, by definition: “32-bit” means that memory addresses in the process are 32 bits wide, and if you have 232 distinct addresses you can address at most 232 bytes (4GB). A 32-bit Linux kernel can only execute 32-bit processes. Depending on the kernel compilation options, each process can only allocate 1GB, 2GB or 3GB ...


4

“Own virtual address space” actually means the opposite of what you say: it means that the address 11111111 points to a different location in physical memory in every process. If virtual addresses corresponded to the same physical address in different processes, then the processes would share an address space. In fact, some of the address space is shared ...


4

The change is reflected immediately. There is no caching along the way. When you read /proc/<pid>/smaps, you actually trigger a traversal of that process's page table. Information about the mappings is accumulated along the way, then displayed, without any caching. The code behind the /proc/<pid>/smaps file is in fs/proc/task_mmu.c, specifically ...



Only top voted, non community-wiki answers of a minimum length are eligible