1

In his response to the question How frequently is the proc file system updated on Linux?, Jonathan Ben-Avraham said that /proc/.../statm is current as soon as you read it since reading it directly triggers kernel callbacks.

What about the data read by those kernel callbacks? Is that always correct or is there some time lag between malloc/new requested memory and the kernel making the measurement available for use by /proc/.../statm ?

I am trying to find a method of measuring the current size of objects allocated via malloc or new.

Running a small test program which allocates some data, neither /proc/.../statm nor calling sbrk() seems to immediately correlate with the amount of memory allocated by the process.

Is there no way to get accurate information?

PROGRAM ALLOCATING 64KB and 128KB chunks

$ ./a.out
MALLOC TEST. Size = 131072
0 ALLOC: HEAP SIZE: 0   MEMORY USAGE (statm): 1083 201 173 2 0 341 0
1 ALLOC: HEAP SIZE: 135168  MEMORY USAGE (statm): 1083 211 182 2 0 341 0
2 ALLOC: HEAP SIZE: 135168  MEMORY USAGE (statm): 1116 215 185 2 0 374 0
3 ALLOC: HEAP SIZE: 135168  MEMORY USAGE (statm): 1149 216 185 2 0 407 0
4 ALLOC: HEAP SIZE: 135168  MEMORY USAGE (statm): 1182 217 185 2 0 440 0
5 ALLOC: HEAP SIZE: 135168  MEMORY USAGE (statm): 1215 218 185 2 0 473 0
6 ALLOC: HEAP SIZE: 135168  MEMORY USAGE (statm): 1248 219 185 2 0 506 0
7 ALLOC: HEAP SIZE: 135168  MEMORY USAGE (statm): 1281 220 185 2 0 539 0
8 ALLOC: HEAP SIZE: 135168  MEMORY USAGE (statm): 1314 221 185 2 0 572 0
9 ALLOC: HEAP SIZE: 135168  MEMORY USAGE (statm): 1347 222 185 2 0 605 0

$ ./a.out
MALLOC TEST. Size = 65536
0 ALLOC: HEAP SIZE: 0   MEMORY USAGE (statm): 1067 201 174 2 0 325 0
1 ALLOC: HEAP SIZE: 0   MEMORY USAGE (statm): 1067 211 182 2 0 325 0
2 ALLOC: HEAP SIZE: 0   MEMORY USAGE (statm): 1067 215 185 2 0 325 0
3 ALLOC: HEAP SIZE: 196608  MEMORY USAGE (statm): 1115 216 185 2 0 373 0
4 ALLOC: HEAP SIZE: 196608  MEMORY USAGE (statm): 1115 217 185 2 0 373 0
5 ALLOC: HEAP SIZE: 196608  MEMORY USAGE (statm): 1115 218 185 2 0 373 0
6 ALLOC: HEAP SIZE: 393216  MEMORY USAGE (statm): 1163 219 185 2 0 421 0
7 ALLOC: HEAP SIZE: 393216  MEMORY USAGE (statm): 1163 220 185 2 0 421 0
8 ALLOC: HEAP SIZE: 393216  MEMORY USAGE (statm): 1163 221 185 2 0 421 0
9 ALLOC: HEAP SIZE: 589824  MEMORY USAGE (statm): 1211 222 185 2 0 469 0

TEST PROGRAM

class CfgProfileList
{
public:
    bool obtainSystemProfileList();
    void leakTest();
    void leakObjTest();
    std::set<std::string> mProfileList;
private:
    char dummy[1024 * 1024]; // use up some space
};

class ComUtil
{
public:
    static void printMemoryUsage();

private:
    static unsigned int mHeapOrigin;
};

/* static */
unsigned int ComUtil::mHeapOrigin = 0;

// Print current process memory utilization
/* static */ void
ComUtil::printMemoryUsage()
{
    unsigned int pHeap = (unsigned int)sbrk(0);
    if (mHeapOrigin == 0)
        mHeapOrigin = pHeap;

    printf("HEAP SIZE: %u\t", pHeap - mHeapOrigin);

    char fname[256], line[256];
    sprintf(fname, "/proc/%d/statm", getpid());

    FILE *pFile = fopen(fname, "r");
    if (!pFile)
        return;

    fgets(line, 255, pFile);
    fclose(pFile);
    printf("MEMORY USAGE (statm): %s", line);
}



void
CfgProfileList::leakTest()
{
    char *pointerList[50];
    int  n = 10;
    int  sleep = 1; 
    int  size  = 64 * 1024;

    printf("MALLOC TEST. Size = %d\n", size);

    for (int i = 0; i < n; i++)
    {
        pointerList[i] = (char *)malloc(size);
        printf("%d ALLOC: ", i);
        ComUtil::printMemoryUsage();
        usleep(sleep);
    }
}

int
main(int argc, char **argv)
{
    CfgProfileList pl;
    pl.leakTest();
}
1

Your traces seem mostly OK to me... (Remember there’s an initial heap which your code doesn’t measure.)

MALLOC TEST. Size = 131072

Here you're allocating 128KB blocks, so the allocator probably won’t use sbrk(), but mmap().

0 ALLOC: HEAP SIZE: 0   MEMORY USAGE (statm): 1083 201 173 2 0 341 0
1 ALLOC: HEAP SIZE: 135168  MEMORY USAGE (statm): 1083 211 182 2 0 341 0

This one’s the odd one, your heap increases but not your program pages.

2 ALLOC: HEAP SIZE: 135168  MEMORY USAGE (statm): 1116 215 185 2 0 374 0
3 ALLOC: HEAP SIZE: 135168  MEMORY USAGE (statm): 1149 216 185 2 0 407 0
4 ALLOC: HEAP SIZE: 135168  MEMORY USAGE (statm): 1182 217 185 2 0 440 0
5 ALLOC: HEAP SIZE: 135168  MEMORY USAGE (statm): 1215 218 185 2 0 473 0
6 ALLOC: HEAP SIZE: 135168  MEMORY USAGE (statm): 1248 219 185 2 0 506 0
7 ALLOC: HEAP SIZE: 135168  MEMORY USAGE (statm): 1281 220 185 2 0 539 0
8 ALLOC: HEAP SIZE: 135168  MEMORY USAGE (statm): 1314 221 185 2 0 572 0
9 ALLOC: HEAP SIZE: 135168  MEMORY USAGE (statm): 1347 222 185 2 0 605 0

Here every allocation eats up an extra 33 pages, or 132KB, off the heap, so everything’s OK.

MALLOC TEST. Size = 65536
0 ALLOC: HEAP SIZE: 0   MEMORY USAGE (statm): 1067 201 174 2 0 325 0
1 ALLOC: HEAP SIZE: 0   MEMORY USAGE (statm): 1067 211 182 2 0 325 0
2 ALLOC: HEAP SIZE: 0   MEMORY USAGE (statm): 1067 215 185 2 0 325 0

Here your allocations fit in the initial heap.

3 ALLOC: HEAP SIZE: 196608  MEMORY USAGE (statm): 1115 216 185 2 0 373 0
4 ALLOC: HEAP SIZE: 196608  MEMORY USAGE (statm): 1115 217 185 2 0 373 0
5 ALLOC: HEAP SIZE: 196608  MEMORY USAGE (statm): 1115 218 185 2 0 373 0

The heap is bumped by 192KB, which matches the allocated pages, and fits three allocations.

6 ALLOC: HEAP SIZE: 393216  MEMORY USAGE (statm): 1163 219 185 2 0 421 0
7 ALLOC: HEAP SIZE: 393216  MEMORY USAGE (statm): 1163 220 185 2 0 421 0
8 ALLOC: HEAP SIZE: 393216  MEMORY USAGE (statm): 1163 221 185 2 0 421 0

Same again...

9 ALLOC: HEAP SIZE: 589824  MEMORY USAGE (statm): 1211 222 185 2 0 469 0

... and again.

Running with strace -e brk,mmap will help understand things. The information available from sbrk() and /proc is accurate, without delay.

  • Thanks, that's very helpful. The docs weren't clear about the 'chunky-ness' of the allocation. – Danny Dec 12 '16 at 15:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.