Tag Info

Hot answers tagged

78

You can write a C program to malloc the required memory and then use mlock() to prevent the memory from being swapped out. Then just let the program wait for keyboard input, and unlock the memory, free the memory and exit.


60

/proc/$pid/maps /proc/$pid/mem shows the contents of $pid's memory mapped the same way as in the process, i.e., the byte at offset x in the pseudo-file is the same as the byte at address x in the process. If an address is unmapped in the process, reading from the corresponding offset in the file returns EIO (Input/output error). For example, since the first ...


56

Vim sometimes has trouble with files that have unusually long lines. It's a text editor, so it's designed for text files, with line lengths that are usually at most a few hundred characters wide. A database file may not contain many newline characters, so it could conceivably be one single 100 Mb long line. Vim will not be happy with that, and although it ...


46

In my experience Vim chokes not on large files, but on long lines. Use this command to have mysqldump use shorter lines at the expense of a larger file: $ mysqldump --complete-insert -u -p Additionally, you can open Vim and ask it not to parse your .vimrc file or load any plugins with this command: $ vim -u NONE output.sql Loading Vim in this manner ...


42

I would suggest running a VM with limited memory and testing the software in that would be a more efficient test than trying to fill memory on the host machine. That method also has the advantage that if the low memory situation causes OOM errors elsewhere and hangs the whole OS, you only hang the VM you are testing in not your machine that you might have ...


42

stress is a workload generator that simulates cpu/mem/io/hdd stress on POSIX systems. This call should do the trick on Linux: stress --vm-bytes $(awk '/MemFree/{printf "%d\n", $2 * 0.9;}' < /proc/meminfo)k --vm-keep -m 1 Adapt the /proc/meminfo call with free(1)/vm_stat(1)/etc. if you need it portable.


39

On a 32-bit architecture, the address space range for addressing RAM is: 0x00000000 - 0xffffffff or 4'294'967'295 (4 GB). The linux kernel splits that up 3/1 (could also be 2/2, or 1/3) into user space (high memory) and kernel space (low memory) respectively. The user space range: 0x00000000 - 0xbfffffff Every newly spawned user process gets an ...


34

It can. There are 2 different out of memory conditions you can encounter in linux. Which you encounter depends on the value of sysctl vm.overcommit_memory (/proc/sys/vm/overcommit_memory) Introduction: The kernel can perform what is called 'memory overcommit'. This is when the kernel allocates programs more memory than is really present in the system. This ...


28

Two potential problems: grep -R (except for the modified GNU grep found on OS/X 10.8 and above) follows symlinks, so even if there's only 100GB of files in ~/Documents, there might still be a symlink to / for instance and you'll end up scanning the whole file system including files like /dev/zero. Use grep -r with newer GNU grep, or use the standard ...


26

run linux; boot with mem=nn[KMG] kernel boot parameter (look in linux/Documentation/kernel-parameters.txt for details).


25

Since you don't mention, I'm assuming this is on Linux. dmidecode -t memory lshw -class memory For RHEL systems, to view their RAM: dmidecode -t 16


24

This exact same question was asked on serverfault just the other day :-) The linux virtual memory system isn't quite so simple. You can't just add up all the RSS fields and get the value reported used by free. There are many reasons for this, but I'll hit a couple of the biggest ones. When a process forks, both the parent and the child will show with the ...


23

Clean pages are pages that have not been modified since they were mapped (typically, text sections from shared libraries are only read from disk (when necessary), never modified, so they'll be in shared, clean pages). Dirty pages are pages that are not clean (i.e. have been modified). Private pages are available only to that process, shared pages are mapped ...


23

If your process doesn't spawn more children that consume the most memory, you may use setrlimit function. More common user interface for that is using ulimit command of a shell: $ ulimit -Sv 500000 # Set ~500 mb limit $ pdftoppm ... This will only limit "virtual" memory of your process, taking into account—and limiting—the memory the process being ...


22

ulimit is made for this. You can setup defaults for ulimit on a per user or a per group basis in /etc/security/limits.conf ulimit -v KBYTES sets max virtual memory size. I don't think you can give a max amount of swap. It's just a limit on the amount of virtual memory the user can use. So you limits.conf would have the line (to a maximum of 4G of ...


21

From inside top you can try the following: Press SHIFT+f Press the Letter corresponding to %MEM Press ENTER You might also try: $ ps -eo pmem,pcpu,vsize,pid,cmd | sort -k 1 -nr | head -5 This will give the top 5 processes by memory usage.


21

From this HN comment: https://news.ycombinator.com/item?id=6695581 Just fill /dev/shm via dd or similar. swapoff -a dd if=/dev/zero of=/dev/shm/fill bs=1k count=1024k


17

What you're asking for is called DMA. You need to write a driver to reserve this memory. Yes, I realize you said you didn't want the OS to intervene, and a driver becomes part of the OS, but in absence of a driver's reservation, the kernel believes all memory belongs to it. (Unless you tell the kernel to ignore the memory block, per Aaron's answer, that ...


17

There's some problems with ulimit. Here's a useful read on the topic: Limiting time and memory consumption of a program in Linux, which lead to the timeout tool, which lets you cage a process (and it's forks) by time or memory consumption.


17

NOTE: I'm going to assume that your machine has a memory mapping unit (MMU). There is a Linux version (µClinux) that doesn't require an MMU, and this answer doesn't apply there. What is an MMU? It's hardware—part of the processor and/or memory controller. Understanding shared library linking doesn't require you to understand exactly how an MMU works, just ...


16

You can also use htop. It's much cooler than top. If you are using Debian or one of its derivatives, then you can install it using sudo apt-get install htop.


15

This is relevant to the Linux kernel; I'm not sure how any Unix kernel handles this. The High Memory is the segment of memory that user-space programs can address. It cannot touch Low Memory. Low Memory is the segment of memory that the Linux kernel can address directly. If the kernel must access High Memory, it has to map it into its own address space ...


15

You can use tstime to measure the highwater memory usage (RSS and virtual) of a process. For example: $ tstime date Tue Aug 16 21:35:02 CEST 2011 Exit status: 0 pid: 31169 (date) started: Tue Aug 16 21:35:02 2011 real 0.017 s, user 0.000 s, sys 0.000s rss 888 kb, vm 9764 kb It also supports a more easy to parse ...


15

If you want the OS to totally ignore it, you need to make a memory hole using "memmap." See this reference. For example, if you want 512M at the 2GB barrier, you can put "memmap=512M$2G" on your kernel command line. You will need to check your dmesg to find a contiguous hole to steal so you don't stomp on any devices; that is specific to your ...


14

ulimit -v, it's a shell builtin, but it should do what you want. I use that in init scripts sometimes: ulimit -v 128k command ulimit -v unlimited It seems however, that you want ways of manipulating the maximum allocatable memory while the program is running, is that correct? Probably something like renice for manipulating the Priority. There is, ...


14

A 32-bit address space means that you have space for 4GB of addresses. Ideally the kernel likes to be able to map all physical memory, all the memory of the current task, and all of its own memory. If physical memory alone takes up all of the available 4GB, that won't work. So physical memory is divided into low memory, which is mapped all the time, and high ...


14

The general solution to test memory is to write a specific pattern like 0xFFFFFFFF to your memory and read it afterwards and compare the result. You can and should of course alter the pattern to discover problems. Some solutions like memtest86+ also generate random patterns and change the direction they use to write to the memory. For more detailed ...


14

Red Hat Enterprise Linux (RHEL) These are probably a good basis, looking at RHEL6's capabilities, they're covered here, titled: Red Hat Enterprise Linux 6 technology capabilities and limits.     NOTE: [5] The architectural limits are based on the capabilities of the Red Hat Enterprise Linux kernel and the physical hardware. Red Hat ...


13

You should read the dmesg values "Memory Akb/Bkb available" as: There is A available for use right now, and the system's highest page frame number multiplied by the page size is B. This is from arch/x86/mm/init_64.c: printk(KERN_INFO "Memory: %luk/%luk available (%ldk kernel code, " "%ldk absent, %ldk reserved, %ldk data, %ldk ...


13

There is a command-line option which does that: -M : Detect memory units Show memory units (k/M/G) and display floating point values in the memory summary. So it is sufficient to run top like that: top -M



Only top voted, non community-wiki answers of a minimum length are eligible