Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I want to do some low-resources testing and for that I need to have 90% of the free memory full.

How can I do this on a *nix system?

share|improve this question
3  
Does it really have to work on any *nix system? –  Michael Kjörling Nov 8 '13 at 12:31
25  
Instead of jut filling memory, could you instead create a VM (using docker, or vagrant, or something similar) that has a limited amount of memory? –  abendigo Nov 8 '13 at 13:27
3  
@abendigo For a QA many of the solutions presented here are useful: for a general purpose OS without a specific platform the VM or kernel boot parameters could be useful, but for a embedded system where you know the memory specification of the targeted system I would go for the filling of the free memory. –  Eduard Florinescu Nov 9 '13 at 17:40
    
In case anyone else is a little shocked by the scoring here: meta.unix.stackexchange.com/questions/1513/…? –  TAFKA 'goldilocks' Nov 13 '13 at 14:46
add comment

12 Answers

up vote 38 down vote accepted

stress is a workload generator that simulates cpu/mem/io/hdd stress on POSIX systems. This call should do the trick on Linux:

stress --vm-bytes $(awk '/MemFree/{printf "%d\n", $2 * 0.9;}' < /proc/meminfo)k --vm-keep -m 1

Adapt the /proc/meminfo call with free(1)/vm_stat(1)/etc. if you need it portable.

share|improve this answer
add comment

You can write a C program to malloc the required memory and then use mlock() to prevent the memory from being swapped out. Then just let the program wait for keyboard input, and unlock the memory, free the memory and exit.

share|improve this answer
17  
Long time back I had to test similar use case. I observed that until you write something to that memory it will not be actually allocated(i.e. until page fault happens) . I am not sure whether mlock() take cares of that. –  siri Nov 8 '13 at 13:31
2  
I concur with @siri; however, it depends on which variant UNIX you are using. –  Anthony Nov 8 '13 at 13:34
1  
Some inspiration for the code. Furthermore, I think you don't need to unlock/free the memory. The OS is going to do that for you when your process has ended. –  Sebastian Nov 8 '13 at 13:44
6  
You probably have to actually write to the memory, the kernel might just overcommit if you only malloc it. If configured to, e.g. Linux will let malloc return successfully without actually having the memory free, and only actually allocate the memory when it is being written to. See win.tue.nl/~aeb/linux/lk/lk-9.html –  bjarkef Nov 8 '13 at 14:32
5  
@Sebastian: calloc will run into the same problem IIRC. All the memory will just point to the same read-only zeroed page. It won't actually get allocated until you try to write to it (which won't work since it is read-only). The only way of being really sure that I know is to do a memset of the whole buffer. See the following answer for more info stackoverflow.com/a/2688522/713554 –  Leo Nov 8 '13 at 16:43
show 2 more comments

I would suggest running a VM with limited memory and testing the software in that would be a more efficient test than trying to fill memory on the host machine.

That method also has the advantage that if the low memory situation causes OOM errors elsewhere and hangs the whole OS, you only hang the VM you are testing in not your machine that you might have other useful processes running on.

Also if your testing is not CPU or IO intensive, you could concurrently run instances of the tests on a family of VMs with a variety of low memory sizes.

share|improve this answer
add comment
  1. run linux;
  2. boot with mem=nn[KMG] kernel boot parameter

(look in linux/Documentation/kernel-parameters.txt for details).

share|improve this answer
add comment

From this HN comment: https://news.ycombinator.com/item?id=6695581

Just fill /dev/shm via dd or similar.

swapoff -a
dd if=/dev/zero of=/dev/shm/fill bs=1k count=1024k
share|improve this answer
7  
Not all *nixes have /dev/shm. Any more portable idea? –  Tadeusz A. Kadłubowski Nov 8 '13 at 12:24
add comment

How about ramfs if it exists? Mount it and copy over a large file? If there's no /dev/shm and no ramfs - I guess a tiny C program that does a large malloc based on some input value? Might have to run it a few times at once on a 32 bit system with a lot of memory.

share|improve this answer
add comment

If you want to test a particular process with limited memory you might be better off using ulimit to restrict the amount of allocatable memory.

share|improve this answer
    
Actually this does not work on linux (dunno about other *nixes). man setrlimit: RLIMIT_RSS Specifies the limit (in pages) of the process's resident set (the number of virtual pages resident in RAM). This limit only has effect in Linux 2.4.x, x < 30, and there only affects calls to madvise(2) specifying MADV_WILLNEED. –  Patrick Nov 8 '13 at 13:46
add comment

I keep a function to do something similar in my dotfiles. https://github.com/sagotsky/.dotfiles/blob/master/.functions#L248

function malloc() {
  if [[ $# -eq 0 || $1 -eq '-h' || $1 -lt 0 ]] ; then
    echo -e "usage: malloc N\n\nAllocate N mb, wait, then release it."
  else 
    N=$(free -m | grep Mem: | awk '{print int($2/10)}')
    if [[ $N -gt $1 ]] ;then 
      N=$1
    fi
    sh -c "MEMBLOB=\$(dd if=/dev/urandom bs=1MB count=$N) ; sleep 1"
  fi
}
share|improve this answer
add comment

How abount a simple python solution?

#!/usr/bin/env python

import sys
import time

if len(sys.argv) != 2:
    print "usage: fillmem <number-of-megabytes>"
    sys.exit()

count = int(sys.argv[1])

megabyte = (0,) * (1024 * 1024 / 8)

data = megabyte * count

while True:
    time.sleep(1)
share|improve this answer
6  
That will probably quickly be swapped out, having very little actual impact on memory pressure (unless you fill up all the swap as well, which will take a while, usually) –  Joachim Sauer Nov 8 '13 at 13:22
    
Why would a unix swap while there is available RAM? This is actually a plausible way to evict disk cache when need be. –  Alexander Shcheblikin Nov 8 '13 at 23:04
    
@AlexanderShcheblikin This question isn't about evicting disk cache (which is useful for performance testing but not for low resources testing). –  Gilles Nov 9 '13 at 14:40
add comment

I think this is a case of asking the wrong question and sanity being drowned out by people competing for the most creative answer. If you only need to simulate OOM conditions, you don't need to fill memory. Just use a custom allocator and have it fail after a certain number of allocations. This approach seems to work well enough for SQLite.

share|improve this answer
add comment

I wrote this little C++ program for that: https://github.com/rmetzger/dynamic-ballooner

The advantage of this implementation is that is periodically checks if it needs to free or re-allocate memory.

share|improve this answer
add comment

Just wondering, why are so many people for going for answers where the person who asked the question would actually have to put their machine to low resources, and in some cases suggesting potentially dangerous ideas? Virtual machines were made for things like this - where you can test your things without risking messing everything up.

Sure, you could go for low resource management by decreasing the amount on your machine, perhaps by forcing it to be used. But you could also just as easily, or perhaps more easily, do it by creating a virtual machine, perhaps on something like VMWare or Virtualbox, decreasing the amount of memory to the bare minimum and running on that. This has the added benefit of also allowing you to test it in a non-dev environment, one closer to the end user, where they may not have all the dependencies pre-installed.

The benefit of this solution therefore I think is obvious - I mean, what if your application decides to not play nice at that level and really cause problems to the computer? You would rather this occur in a virtual environment than your dev environment, I would say.

A further benefit of this would be that you could also take a snapshot of the machine when you tested the program, which would mean that you could test it many times with very little waiting time, while if you tested it on your dev machine, it could take a while before and after to revert to normal. Note that this would also allow you to test a lot of others things - you could easily test the code, for free I might add, on Windows systems as well, and also implement things such as how it handles isolation from the internet, or other such things.

Honestly, all the rest of you should also be considering this - testing on your own system is both wasteful, as you are likely to have dependencies that the end user may not implicitly have (I figured this out when I was setting up my own Jenkins build server), and dangerous (I mean fork bombing - really? I doubt that is a reliable way to do low-resource testing).

EDIT:

As suggested by @slm, you can change the ammount of memory available when creating a VM - for example, in Virtual Box, when creating, you need to specify the ammount of memory that you wish to allocate (you can even have this set dynamically I think). This can then be changed at a later time - so you might create your machine with 1 GB of RAM dedicated to it so that it installs quickly, but when testing can change the ammount of RAM in the settings tab to whatever you want - and then take a snapshot of this ammount. I personally wanted to test this a while back, and managed to determine the bare minimum of actual RAM that my programme could safely work with - and then used that exact snapshot (which retains the software and hardware specs) for further testing later to allow me to test later releases of the software I was making against the same conditions.

By the way, you may also want to consider using a CI solution to do this - running tests manually yourself can take quite a while, and distracts away from actual coding time. If you are willing to spend some money, you could look at Microsoft's ALM solution - Team Foundation Server, through which you can run all sorts of tests such as Unit, Integration, Load etc, along with actually letting the software start up "lab machines" - aka virtual machines on the hardware and testing on that. My friend worked at a software company where they ran that everyday - they would upload their code to the server, and every night at midnight, the server would run through the code and do the coded tests, then startup a virtual machine from a particular snapshot and test it, then submit a report. If you are not willing to spend a lot on this, then there are a couple of free solutions as well, like Jenkins and TeamCity, through which you may be able to set this up as well.

share|improve this answer
    
How is this any different than the @DavidSpillett answer? –  slm Nov 9 '13 at 3:15
    
Sorry, didn't see his one - seems like he bet me too it, haha. I did run a search for "virtual" on this page, but not for VM unfortunately. By the way, thank you for the editing. I was in a rush, and so only had time to type out what I wanted, and so it may have looked quite messy –  Sbspider Nov 9 '13 at 3:15
    
Yeah I figured you didn't I also cleaned up your A too. If there is a different angle you can take yours I'll gladly UV it but it isn't saying anything different than his, at the moment. So the DV was more to push this to the bottom than to say it was necessarily a weak answer by itself. –  slm Nov 9 '13 at 3:17
    
I'll see what I can add. –  Sbspider Nov 9 '13 at 3:18
2  
Haha. I'll add that then. And you are right - no Q should have this many, or at least not this many bad ones. I mean, fork-bombing - really? Last time I checked, that is not a legit way to actually test your code before releasing it, unless you want to risk destroying everything, including what you are writing –  Sbspider Nov 9 '13 at 3:33
show 3 more comments

protected by Gilles Nov 9 '13 at 14:37

Thank you for your interest in this question. Because it has attracted low-quality answers, posting an answer now requires 10 reputation on this site.

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.