I have a group of programs that I was stress testing today. I drastically shot up the input load to the program and it worked at first, but I had an apparent memory leak sucking up memory. The only problem is that 1) Valgrind is insisting I don't have any leaks and 2) it only happens a high packet ingest rates.

One, possible crazy, idea was that the extra memory may be going towards a message queue due to my system lacking the bandwidth to output packets as fast as they are incoming forcing increasingly large buffers of data to be transmitted. The program that is showing as hogging the memory is reading from TCP socket and writing to a named pipe. so I have three questions.

  1. Can the in-memory queue for storing to-be dispatched data increase indefinately for either TCP/IP or named pipes?

  2. if the queue can and did grow would it still show that My program (the one reading from TCP and outputting to the named pipe) was the memory hog, or would the memory just show as system memory?

  3. is there a command I can use to check memory allocated to these queues? I'm running Centos

Admitedly I don't really think this is the cause of my leak, but then again I realy can't guess what else is so I have to ask!

1 Answer 1

  1. Generally its quite bounded. Though there are two places (at least) these buffers can exist:
    1. In the kernel. Definitely bounded. FIFO is hard-coded I believe, TCP can be set in /proc/sys/kernel/tcp_wmem (and rmem), and (within the proc limits) with setsockopt SO_SNDBUF/SO_RCVBUF. See the tcp(7) manpage for details.
    2. In your app. If you're not calling write, etc. directly, the library you're using may have its own buffers. These may be infinite.
    3. Ok. There is actually a third place, in RAM on your NIC. Can not be increased beyond a certain (fairly small) limit without soldering, so I'm ignoring this. Oh, and there is a transmit ring buffer, for your NIC to DMA out of. Again, fixed-size (ethtool to tune, if even possible).
  2. The kernel ones show as system memory. The ones in your app will show as part of your app's virtual size (and resident size, etc.)
  3. netstat -t will show you the current size of the send and receive queue for each TCP connection. For ones held in your library, check its documentation.

Remember, memory isn't leaked if you eventually free it. If you're just suffering from ever-growing queues in your app, that's not leaked. Same if you're suffering from ever-growing memory fragmentation.

  • I expected, and yet was hopping not, to get this answer. If it was the buffers I would at least have someplace to look at. Oh well. Thanks for your answer.
    – dsollen
    Aug 29, 2012 at 17:18
  • Does top show this as buffer-memory (sum)?
    – Nils
    Aug 29, 2012 at 21:35
  • @Nils I think buffer memory in top is filesystem buffers, not network.
    – derobert
    Aug 29, 2012 at 21:36

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .