Hi Unix/Linux Gurus out there!
I need a solid proof that TIME_WAIT (a lot of it actually) is the real culprit in the slowdown in one of our servers. The server is hosted on Parallels Baremetal virtualization, and the actual server is a VM: CentOS5 with dual CPU and 2GB RAM.
A week ago, we started to notice that it was too slow that even doing an 'ls' on a directory with just a few files in there (around 20) would need around 1.5 seconds to display the results.
I tried doing
vmstat but it doesn't seem to be even using it's swap. No bottlenecks on the network. But running
top, you'd see java mostly hogging the resource. Java is needed since this VM is our hudson server.
One of my colleagues tried checking the connections via
$ vmstat -vatpno
And noticed that there where a lot of connections in TIME_WAIT...around 300+. So we tried applying some of the recommendations in this page particularly that of TCP_FIN_TIMEOUT, TCP_KEEPALIVE_INTERVAL & TCP_KEEPALIVE_PROBES. The connections in TIME_WAIT went lower but still fluctuates between 220 to 280(maybe due to the fact that a new connection is added from time to time and other connections in TIME_WAIT is not yet "timed-out"). Perhaps we could try adding TCP_TW_RECYCLE & TCP_TW_REUSE later when we don't see any improvement.
Now going back to my main question: is there a solid evidence that a lot of TIME_WAIT'ed connections eat up a lot of RAM?
Thanks in advance.