Finding the PID of an established connection is trivial using netstat or lsof. However, I have a process which is creating a connection ever 60 seconds to our database and locking it up by maxing out the failed connection attempt limit. I can increase the failed connection limit to something extremely high on the database, or I can try to track down what is making the connection, and I have chosen the latter.

Based on tcpdump/wireshark, I can see that what is happening is that a connection is established and then the connecting server immediately closes the connection before the server can even respond. What I don't know is why.

The first step is to find out what PID is opening the connection. Unfortunately, this seems easier said than done. The problem is that when a connection goes into TIME_WAIT state, it is no longer associated with a PID. Since my connection has a lifetime of less than a tenth of a second, is there any way to record this information?

netstat and lsof appear to be able to poll every second, but this simply isn't fast enough with the connection attempt I am dealing with. Is there a hook that I can connect to to dump this information to a log? Or is my only option to brute force it with a loop and some coding?

I use CentOS 6.

  • What OS do you have? Linux have SystemTap while FreeBSD/Solaris derivatives have DTrace that can achieve that very easily.
    – myaut
    Feb 26, 2015 at 6:04
  • centos 6.x [padding to allow me to actually post this because my real answer was too short] Feb 26, 2015 at 6:24

1 Answer 1


Consider using SystemTap. It is dynamic instrumenting engine that dynamically patches kernel so you can track any in-kernel event such as opening a socket. It is actively developed by RedHat so it is supported in CentOS.


To install SystemTap on CentOS 6:

  1. Enable debuginfo repository:

    sed -i 's/^enabled=0/enabled=1/' /etc/yum.repos.d/CentOS-Debuginfo.repo
  2. Install SystemTap:

    yum install systemtap
  3. Install debuginfo packages for kernel. It can be done manually, but there is a tool that can do it automatically:



SystemTap has not tapset probe for TCP connection, but you may directly bind to kernel functions! You can also do it on socket level.

I.e. create script called conn.stp:

probe kernel.function("tcp_v4_connect") {
    printf("connect [%s:%d] -> %s:%d\n", execname(), pid(),
            ip_ntop(@cast($uaddr, "struct sockaddr_in")->sin_addr->s_addr),
            ntohs(@cast($uaddr, "struct sockaddr_in")->sin_port));

This will give you the following output:

# stap conn.stp
connect [nc:2552] ->
connect [nc:2554] ->
connect [nc:2556] ->

However tracking disconnection events seem to be more trickier.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .