Tag Info

Hot answers tagged

23

What might be happening if a process is "killed due to low RAM"? It's sometimes said that linux by default never denies requests for more memory from application code -- e.g. malloc().1 This is not in fact true; the default uses a heuristic whereby Obvious overcommits of address space are refused. Used for a typical system. It ...


19

Clean pages are pages that have not been modified since they were mapped (typically, text sections from shared libraries are only read from disk (when necessary), never modified, so they'll be in shared, clean pages). Dirty pages are pages that are not clean (i.e. have been modified). Private pages are available only to that process, shared pages are mapped ...


14

This won't happen to you if you only ever load 1G of data into memory. What if you load much much more? For example, I often work with huge files containing millions of probabilities which need to be loaded into R. This takes about 16GB of RAM. Running the above process on my laptop will cause it to start swapping like crazy as soon as my 8GB of RAM have ...


10

It isn't sticky - you just write to the file to make it drop the caches and then it immediately starts caching again. Basically when you write to that file you aren't really changing a setting, you are issuing a command to the kernel. The kernel acts on that command (by dropping the caches) then carries on as before.


5

Edit: This answer is incorrect. Though still a possible cause for oom-killer to be invoked, it is not the cause in this specific case. It looks like this is due to memory fragmentation. From the output you provided, the highest order contiguous memory block you have is a 32kb block in the normal zone. This means that if anything tries to allocate a chunk ...


4

You can run swapon -s to see what devices and files are being used for swap. For example, my scientific linux machine says: [user@sl6.3 ~]$ swapon -s Filename Type Size Used Priority /dev/sda3 partition 8388600 833408 -1 So I'm using /dev/sda3 for swap. Also note the priority field that can be ...


4

32-bit processes can only allocate up to 1, 2, 3, or about 4GB, depending on which memory split was chosen when the 32-bit kernel was built. 32-bit processes on a 64-bit kernel can allocate about 4GB. 64-bit processes on a 64-bit x86-64 kernel can allocate up to 128TiB.


4

As far as I understand your question it happens usually in such way: If you allocate memory: Mark memory as allocated but don't allocate physical memory (hence on access there will be page fault). In Linux it stops at this stage but it is possible that system may allocate physical space immediately - then it performs similar algorithm at the end as on ...


4

Processes aren't killed when there is no more RAM, they are killed when they have been cheated this way: Linux kernel commonly allows processes to allocate (i.e. reserve) an amount of virtual memory which is larger than what is really available (part of RAM + all the swap area) as long as the processes only access a subset of the pages they have reserved, ...


4

Physical Address Extension (PAE) sounds exactly like what he's referring to. A 32-bit CPU can only map ~4gb of memory, even if the system has more. But with PAE, you can use >4gb, though only 4gb of it is mapped at any one time (a single process will never be able to use >4gb). So basically when the kernel changes the actively running process, it re-maps ...


3

If you use DISM, make sure you have ample room in your swap. When you shmat an SHM segment with SHM_SHARE_MMU (which is not the default), you get an ISM segment, which is automatically locked in memory (not pageable). The cost of that mapping, in virtual memory, is just the size of the allocated SHM region. (Since it cannot be paged out, no need to reserve ...


3

swapon have -p switch which sets the priority. I can set up: swapon -p 32767 /dev/zram0 swapon -p 0 /dev/my-lvm-volume/swap Or in /etc/fstab: /dev/zram0 none swap sw,pri=32767 0 0 /dev/my-lvm-volume/swap none swap sw,pri=0 0 0 EDIT: Just for a full solution - such line may be helpful as udev rule: KERNEL=="zram0", ACTION=="add", ...


3

I admit that the following isn't a great answer, but I believe the 0x8048000 value is enshrined in the ELF Specification. See figures A.4, A.5 and A.6 in that doc. The System V ABI Intel 386 Architecture Supplement also standardizes on 0x8048000. See page 3-22, Figue 3-25. 0x804800 is prescribed as the low text segment address/high stack address. And ...


3

A 32-bit process has a 32-bit address space, by definition: “32-bit” means that memory addresses in the process are 32 bits wide, and if you have 232 distinct addresses you can address at most 232 bytes (4GB). A 32-bit Linux kernel can only execute 32-bit processes. Depending on the kernel compilation options, each process can only allocate 1GB, 2GB or 3GB ...


3

Linux as well as Windows, work pretty much the same here. Every process gets it's own "virtual" address space. This doesn't mean that the memory is actually physically available (obviously most 32bit computers never had enough memory), that's, why it's virtual. Also the addresses used there don't correspond to the physical addresses. Thereby physical memory ...


3

Based on the question and comments, you could look up the java running call stack via the jstack command: jstack processid If there are some threads waiting for a long time on some condition then it is most likely a deadlock. A deadlock might be rare on production grade code but common on experimental multithreaded code. In the former case, a rerun might ...


2

Virtual addresses are only backed by physical memory and swap when required. A process is welcome to allocate as much memory as it likes, and the system is welcome to not actually give it that memory until it is needed. See the malloc(3) man page, NOTES section for details.


2

3 > drop_caches instructs the kernel to discard all cached data (that are not needed any more). swapoff will try to pull as much of data, that are currently on swap, as it can back into memory. It may also trigger dropping some cached pages to make room for what is coming into memory from the swap file. You really only seldom need to do this, the kernel ...


2

“Own virtual address space” actually means the opposite of what you say: it means that the address 11111111 points to a different location in physical memory in every process. If virtual addresses corresponded to the same physical address in different processes, then the processes would share an address space. In fact, some of the address space is shared ...


2

From the kernel documentation, in Documentation/x86/x86_64/mm.txt: Virtual memory map with 4 level page tables: 0000000000000000 - 00007fffffffffff (=47 bits) user space, different per mm 247 bytes = 128TiB


2

Serge answered it. The TLB has a fixed number of slots. If a virtual address can be mapped to a physical address with information in the TLB, you avoid an expensive page table walk. But the TLB cannot cache mappings for all pages. Therefore, if you use larger pages, that fixed number of virtual to physical mappings covers a greater overall address range, ...


2

It seems that Linux kernel handle TLB and TLB cache with the same approach. At the architecture independent level there is no thing like flushing a part or entire TLB cache AFAIK. The vm will for example page out things, and there are some hooks to this action which will trigger hardware dependent code if necessary. Depending on the CPU, the kernel might ...


2

You probably want to look at Transparent Hugepages. The .config item is CONFIG_TRANSPARENT_HUGEPAGE. Note that enabling this won't give you huge pages automatically. You'll need to set the CONFIG_TRANSPARENT_HUGEPAGE_MADVISE to 'n', in order to make it the default. Also note that this doesn't allow you to choose an arbitrary page size. I allows to use the ...


2

Pages of process memory may be displaced from the RAM to the disk. This is called swapping or paging (the terms are essentially synonymous). The data is moved to the swap space, and loaded back from the swap space when it is needed. Linux supports both partitions (and other block devices) and files as swap space. If the page in question contains data that's ...


2

As far as I know, you cannot disable the concept of virtual memory in Linux, at least not without totally rewriting it. It's an integral part of the memory management and lots of stuff simply would cease to work if you could disable it. The mmap call can be used to map a file or a device to a part of an application's virtual memory (every application has 4 ...


2

On some demand-paged virtual memory systems, the operating system refuses to allocate anonymous pages (i.e. pages containing data without a filesystem source such as runtime data, program stack etc.) unless there is sufficient swap space to swap out the pages in order to free up physical memory. This strict accounting has the advantage that each process is ...


2

The best way I can attempt to answer those questions is to say what those three actually are. zRAM zRAM is nothing more than a swap device in essence. The memory management will push pages out to the swap device and zRAM will compress that data, allocating memory as needed. Zswap Zswap is a compressed swap space that is allocated internally by the kernel ...


1

You need to be root or the user that the process is owned by to use that command. Try this instead: $ sudo pmap -d 23440 For example $ sudo pmap -d 1457 1457: /usr/sbin/httpd Address Kbytes Mode Offset Device Mapping 00007ff9f23bf000 76 r-x-- 0000000000000000 0fd:00000 zip.so 00007ff9f23d2000 2044 ----- 0000000000013000 ...


1

This answer is for a IA-32 architecture. I took the information form Intels IA-32 Architectures Software Developer’s Manuals, Page 1751/3044(!): Table 4-6. Format of a 32-Bit Page-Table Entry that Maps a 4-KByte Page: 0 (P): Present; must be 1 to map a 4-KByte page 1 (R/W): Read/write; if 0, writes may not be allowed to the 4-KByte page referenced by this ...



Only top voted, non community-wiki answers of a minimum length are eligible