Tag Info

Hot answers tagged

25

What might be happening if a process is "killed due to low RAM"? It's sometimes said that linux by default never denies requests for more memory from application code -- e.g. malloc().1 This is not in fact true; the default uses a heuristic whereby Obvious overcommits of address space are refused. Used for a typical system. It ...


23

Clean pages are pages that have not been modified since they were mapped (typically, text sections from shared libraries are only read from disk (when necessary), never modified, so they'll be in shared, clean pages). Dirty pages are pages that are not clean (i.e. have been modified). Private pages are available only to that process, shared pages are mapped ...


14

This won't happen to you if you only ever load 1G of data into memory. What if you load much much more? For example, I often work with huge files containing millions of probabilities which need to be loaded into R. This takes about 16GB of RAM. Running the above process on my laptop will cause it to start swapping like crazy as soon as my 8GB of RAM have ...


12

It isn't sticky - you just write to the file to make it drop the caches and then it immediately starts caching again. Basically when you write to that file you aren't really changing a setting, you are issuing a command to the kernel. The kernel acts on that command (by dropping the caches) then carries on as before.


8

The text segment is the mapping at 0x400000 - it's marked 'r-x' for readable and executable. The mapping at 0x600000 is read-only, so that's almost certainly the ".rodata" section of the executable file. GCC puts C string literals into a read-only section. The mapping at 0x601000 is 'rw-', so that's probably the famed heap. You could have your executable ...


6

Well. How's that for fried RAM? Guess that was the culprit, after all. I'm pleased to report that, after removing the defective stick, everything is going quite smoothly.


6

It appears that the stack memory limit is not allocated (anyway, it couldn't with unlimited stack). https://www.kernel.org/doc/Documentation/vm/overcommit-accounting says: The C language stack growth does an implicit mremap. If you want absolute guarantees and run close to the edge you MUST mmap your stack for the largest size you think you will ...


6

Both. A page is the smallest unit of memory that the hardware page mapping function of the CPU deals with. If for example, pages are 4096 bytes in size, then each page begins and ends on a boundary aligned to 4096 bytes in both physical and virtual memory and all 4096 bytes in the page are mapped linearly and contiguously from virtual address to physical ...


5

From the kernel documentation, in Documentation/x86/x86_64/mm.txt: Virtual memory map with 4 level page tables: 0000000000000000 - 00007fffffffffff (=47 bits) user space, different per mm 247 bytes = 128TiB


5

32-bit processes can only allocate up to 1, 2, 3, or about 4GB, depending on which memory split was chosen when the 32-bit kernel was built. 32-bit processes on a 64-bit kernel can allocate about 4GB. 64-bit processes on a 64-bit x86-64 kernel can allocate up to 128TiB.


5

Edit: This answer is incorrect. Though still a possible cause for oom-killer to be invoked, it is not the cause in this specific case. It looks like this is due to memory fragmentation. From the output you provided, the highest order contiguous memory block you have is a 32kb block in the normal zone. This means that if anything tries to allocate a chunk ...


5

The run-time library calls sbrk to read the current end of the data segment. sbrk is implemented on top of the brk syscall; as you can see, calling brk with an invalid address just returns the current value.


4

As far as I understand your question it happens usually in such way: If you allocate memory: Mark memory as allocated but don't allocate physical memory (hence on access there will be page fault). In Linux it stops at this stage but it is possible that system may allocate physical space immediately - then it performs similar algorithm at the end as on ...


4

A 32-bit process has a 32-bit address space, by definition: “32-bit” means that memory addresses in the process are 32 bits wide, and if you have 232 distinct addresses you can address at most 232 bytes (4GB). A 32-bit Linux kernel can only execute 32-bit processes. Depending on the kernel compilation options, each process can only allocate 1GB, 2GB or 3GB ...


4

You can run swapon -s to see what devices and files are being used for swap. For example, my scientific linux machine says: [user@sl6.3 ~]$ swapon -s Filename Type Size Used Priority /dev/sda3 partition 8388600 833408 -1 So I'm using /dev/sda3 for swap. Also note the priority field that can be ...


4

Processes aren't killed when there is no more RAM, they are killed when they have been cheated this way: Linux kernel commonly allows processes to allocate (i.e. reserve) an amount of virtual memory which is larger than what is really available (part of RAM + all the swap area) as long as the processes only access a subset of the pages they have reserved, ...


4

Physical Address Extension (PAE) sounds exactly like what he's referring to. A 32-bit CPU can only map ~4gb of memory, even if the system has more. But with PAE, you can use >4gb, though only 4gb of it is mapped at any one time (a single process will never be able to use >4gb). So basically when the kernel changes the actively running process, it re-maps ...


4

Huge pages are for allocating chunks of memory, not writing them. Normally when applications need large amounts of memory, they have to allocate many "pages". A page is simply a chunk of physical memory. Normally this chunk is only a few KB. So when an application is doing a lot of memory intensive operations that span many pages, it's expensive for the ...


4

There is more than one definition of the chunk size for memory writes. You could consider it to be: the width of the store instruction (store byte, store word, …), typically 1, 2, 4, 8 or 16; the width of a cache line, typically something like 16 or 64 bytes (and different cache levels may have different line widths); the width of the memory bus, which is ...


3

Embedded systems running on flash don't use swap. Please don't confuse that with virtual memory though - virtual memory has many more usages then just swapping to disk.


3

3 > drop_caches instructs the kernel to discard all the cached data (that are not needed any more). swapoff will try to pull as much of the data, that are currently on swap, as it can back into memory. It may also trigger dropping some cached pages to make room for what is coming into memory from the swap file. You really only seldom need to do this, ...


3

Linux as well as Windows, work pretty much the same here. Every process gets it's own "virtual" address space. This doesn't mean that the memory is actually physically available (obviously most 32bit computers never had enough memory), that's, why it's virtual. Also the addresses used there don't correspond to the physical addresses. Thereby physical memory ...


3

I admit that the following isn't a great answer, but I believe the 0x8048000 value is enshrined in the ELF Specification. See figures A.4, A.5 and A.6 in that doc. The System V ABI Intel 386 Architecture Supplement also standardizes on 0x8048000. See page 3-22, Figue 3-25. 0x804800 is prescribed as the low text segment address/high stack address. And ...


3

Like the docs say, user space gets 247 bytes = 128TiB, and kernel gets 512MiB. The rest of the address space goes to various parts of the system, along with a few unusable holes.


3

If you use DISM, make sure you have ample room in your swap. When you shmat an SHM segment with SHM_SHARE_MMU (which is not the default), you get an ISM segment, which is automatically locked in memory (not pageable). The cost of that mapping, in virtual memory, is just the size of the allocated SHM region. (Since it cannot be paged out, no need to reserve ...


3

swapon have -p switch which sets the priority. I can set up: swapon -p 32767 /dev/zram0 swapon -p 0 /dev/my-lvm-volume/swap Or in /etc/fstab: /dev/zram0 none swap sw,pri=32767 0 0 /dev/my-lvm-volume/swap none swap sw,pri=0 0 0 EDIT: Just for a full solution - such line may be helpful as udev rule: KERNEL=="zram0", ACTION=="add", ...


3

Serge answered it. The TLB has a fixed number of slots. If a virtual address can be mapped to a physical address with information in the TLB, you avoid an expensive page table walk. But the TLB cannot cache mappings for all pages. Therefore, if you use larger pages, that fixed number of virtual to physical mappings covers a greater overall address range, ...


3

The best way I can attempt to answer those questions is to say what those three actually are. zRAM zRAM is nothing more than a swap device in essence. The memory management will push pages out to the swap device and zRAM will compress that data, allocating memory as needed. Zswap Zswap is a compressed swap space that is allocated internally by the kernel ...


3

Based on the question and comments, you could look up the java running call stack via the jstack command: jstack processid If there are some threads waiting for a long time on some condition then it is most likely a deadlock. A deadlock might be rare on production grade code but common on experimental multithreaded code. In the former case, a rerun might ...


3

if you have spare memory increasing memory not working you should change the swappiness parameter. it tells the kernel how often use non physical memory (swap). check swappiness value: cat /proc/sys/vm/swappiness change the swappiness value temporary (lost on reboot) to value 10 (good value that decreases swap usage) sudo sysctl vm.swappiness=10 if you ...



Only top voted, non community-wiki answers of a minimum length are eligible