Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I learned that on Linux the glibc’s malloc() uses mmap() for very big chunk of memories and brk() is used for small allocations. However that does not seem to be the case when I experiment the theory with a test program.

First, I use a ‘for’ loop in my code to assign a big chunk of memory (80MB) implemented by an array with the condition Option1==TRUE, and each allocation is a multiple of a page size. The chunk of memory will be freed altogether when Option2==TRUE. Note Option1 and Option2 can be set from the command line where my process is running.

Observation: when Option1 was satisfied, the RSS value of my process increased 80MB. After that I set Option2 to be TRUE (while Option1 is disatisfied), then about the same amount memory was reduded from the RSS value (given back to OS). Everything is good so far.

After that, I modified the 'for' loop to assign two big chunks of memory (80MB each) implemented by chunk and chunk1 with Option1. Within each loop the same indexes of the two arrays are assigned with the same amount of memory, and each allocation is smaller than one page. The chunk of memory for chunk will be freed completely when Option2==TRUE. And the chunk of memory for chunk1 will be freed completely when Option3==TRUE.

Observation: the RSS value of my process increased 160MB with Option1==TRUE, and the value remained after Option2==TRUE. And it also remained after Option3==TRUE. I use the Valgrind massif tool to check the memory usage and it reported there are about 160 MB pages mmap’ed by the two malloc() functions in the ‘for’ loop.

I can understand that the memory blocks by small allocations are kept in the process so that they can be used for future allocations. But shouldn't those small allocations be implemented by brk() but not mmap()? Is it because valgrind does not provide the accurate report or because there are some other cases that use mmap() to implement malloc() ?

My code for the second test is copied below:

        char *chunk[200000];
        char *chunk1[200000];
        int i = 0;
        int j,k;
......
        if (<Option1 == TRUE>) {
            if (i < 100) {
                for (k=0; k<2000; k++) {
                chunk[i*2000+k] = (char *)malloc(400);
                chunk1[i*2000+k] = (char *)malloc(400);
                memset (chunk[i*2000+k], 0, 400);
                memset (chunk1[i*2000+k], 0, 400);
                }
            }
        }
        if (<Option2 == TRUE>) {
            for (j = 0; j < 200000; j++) {
                if (chunk[j] != NULL) {
                    free(chunk[j]);
                    chunk[j] = NULL;
                }
            }
        }
        if (<Option3 == TRUE>) {
            for (j = 0; j < 200000; j++) {
                if (chunk1[j] != NULL) {
                    free(chunk1[j]);
                    chunk1[j] = NULL;
                }
            }
        }
......
share|improve this question
    
Use strace to see the actual system calls your process issues. Valgrind could interfere with it. Also look at /proc/<pid>/maps or smaps while your process is paused (and not run with valgrind). –  Mat Nov 5 '12 at 16:34

1 Answer 1

It really depends on the malloc implementation. It may pre-allocate a bigger continuous block of memory and then do some magic on top of it - e.g. allocate smaller blocks of memory from different parts of the big block depending on the requested sizes. Such a bigger block can of course be allocated through mmap.

You can check for example jemalloc's source to get an idea how this can work.

share|improve this answer
    
Thanks peterph! –  michelle Nov 7 '12 at 14:47

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.