Get source IP from reverse SSH tunnel

Question

If I use a client to create a reverse tunnel with SSH using -R I will get this output on the server if I run netstat:

default@debian:~$ sudo netstat -pln
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      801/sshd
tcp        0      0 127.0.0.1:39963         0.0.0.0:*               LISTEN      1074/sshd: anonymou
tcp6       0      0 :::22                   :::*                    LISTEN      801/sshd
udp        0      0 0.0.0.0:68              0.0.0.0:*                           451/dhclient

I would like to know what is the IP of the client that created the listening socket on port 39963.

What command can I run so I not only get the bound IP:PORT but the IP of the client that created the tunnel?

Answer 1

I can't give you a single command but there are a few commands which will let you find the client. The problem is with the way OpenSSH creates processes. The process which owns the listening socket 39963 is the child of the process which owns the SSH connection. So they will have different PIDs. In your example you need to find the parent process for 1074. The following works for OpenSSH server on Ubuntu. I can't guarantee every distribution will behave the same way.

Start by finding the PID for the client connection. Use grep to filter the output of netstat.

$ sudo netstat -ptln | grep 39963
tcp        0      0 127.0.0.1:39963         0.0.0.0:*               LISTEN      21921/sshd: philip@
tcp6       0      0 ::1:39963               :::*                    LISTEN      21921/sshd: philip@

The listening process here is 21921. You need it's parent:

$ ps -ef | grep 21921
philip   21921 21919  0 11:01 ?        00:00:00 sshd: philip@pts/1
philip   21924 21921  0 11:01 pts/1    00:00:00 -bash
philip   22844 20309  0 11:15 pts/0    00:00:00 grep --color=auto 21921

# Or try this:

$ ps -ef | awk '$2 == 21921 { print $3 }'
21919

The parent of 21921 here is 21919. Now we look back to netstat to find the client:

$ sudo netstat -ptn | grep 21919
tcp        0      0 192.168.1.36:22          192.168.1.10:54425   ESTABLISHED 21919/sshd: philip

This shows the remote ip is 192.168.1.10

Answer 2

The SSH client and server exchange messages to set up forwardings, but the kernel network table cares/knows only about its local IP stack, so it sees only sshd.

But, OpenSSH also shares some details in the environment (which too is passed via a message exchange).

 sshcpid=$(pgrep -P 1074)               # fetch child PID
 xargs -0 -L1 -a /proc/$sshcpid/environ # show environment strings

The variables SSH_CLIENT and SSH_CONNECTION should both contain what you need.

Here, xargs is a handy way to process the NUL delimited strings in environ.

This assumes:

• linux, for a /proc with environ

• OpenSSH with a process chain like:

  daemon sshd → forked connection sshd → privsep sshd → user shell

e.g. (from pstree -lp):

 init(1)-+
         .
         |-sshd(1015) +-sshd(1072)---sshd(1074)---bash(1075)

The privsep sshd process is the one you see in netstat, the environment you read is from its immediate descendent, the shell (bash in this example).

Answer 3

The lsof command in Linux displays in its output information about files that are opened by processes

Taking the process id of ssh from the netstat,

lsof -p $pid -a -d 3

will show IP of other end of the connection. As ssh can be hopped, this might not be the final location.

$pid is process id of your ssh service

if you don't have lsof you can install it with apt-get install lsof

Answer 4

As has been mentioned, you need to wait until a connection has been established before you can see the foreign address (on the remote side) of the connection. You could go about getting this information in two ways:

• Either sample netstat periodically, and then grep out the relevant IP:PORT. Straightforward, but you wouldn't catch short lived connections.
• Your other alternative is to setup a dedicated log with iptables and then process this. See e.g. Have iptables run command/script on new connection attempt.

Either method runs asynchronously, and you would need to maintain the connection processing separate from ssh.

Answer 5

Based on top answer idea, I wrote a simple script to automate the process. The single argument is the listening port, and it returns the ip and outgoing port of the client.

#!/bin/bash
PORT_TUNEL=$1
PID=`fuser $PORT_TUNEL/tcp 2>/dev/null`
# xargs is just used for trimming
PARENT_PID=`ps -p ${PID:-$$} -o ppid= | xargs`
netstat -ptn |  awk -v ppid="$PARENT_PID/sshd:"  '{if($7==ppid) print $5}'

Usage:

rsship 39963

Sample output:

67.214.221.43:26412

Explanation:

This script asumes you already know which listening port you want to obtain the client's IP. As explained in other answers, it is directly impossible to know that from the connection itself. Instead we need to obtain the process id (pid) that is holding the listening port. We're using fuser in order to do that and storing it in a variable. Since fuser has some output in stderr that would mess up with our code, we redirect it to /dev/null. Now we have the PID that holds the listening port in our $PID variable.

The next step it to get the parent process, which is the process maintaining the SSH connection with your client. It uses ps in order to do that. Xargs is used to trim the output, as it may come with some blankspaces that would mess up with our awk later.

At last, it uses netstat to get a list of all the connections, and awk to display column 5 when column 7 matches the string "$PARENT_PID/sshd:". Note that -v option is used to pass a bash variable to awk in the proper way. I'm not going to extend myself on this, search "passing bash variables to awk script" if you need more insights abouts this. I'm declaring ppid variable, holding "$PARENT_PID/sshd:", which is the text I want to match against the 7th col. We're using this instead of grep because 1150/sshd: would match 150/sshd:. There are multiple ways of doing this, but this awk command was the simplest and most readable I could come up with. This comes after some struggles because greping just $PARENT_PID often matched some random ports on diferent connections giving unexpected results.

I'm aware this could be written in a much more compact way, but I like to keep the scripts readable and self-explanatory in a glipse when it makes sense

Listening ports enumerator

I've a script to list ALL listenings reverse tunnels on my machine and fetch their client's IP and outgoing port:

In my setup I have a special SSH server for this purpose, so all connections come through "sshtunnel" user. You'll have to modify the script if you're using your normal ssh server and come up with another way to get your first list of listening ports. using "grep sshd" will ALMOST do the trick, although it will list your main shh server listening ports too, although in MY particular situation it works because ssh server was bind to *, so the grep 127.0.0.1 would filter it out too:

#!/bin/bash

IP_SCRIPT="/etc/openvpn/rsship"

PORTS=`lsof -i -P -n | grep LISTEN | grep sshtunnel | grep 127.0.0.1 | awk -F" " '{print $9}' | awk -F":" '{print $2}' | awk '{if($1==$1+0 && $1>16000 && $1<16255) print $1}' | sort -n`


while IFS= read -r line; do
    REAL_IP=$($IP_SCRIPT $line)
    echo $line$'\t'$REAL_IP
done <<< "$PORTS"

Note: $IP_SCRIPT references to the first script posted.

grep 127.0.0.1 is used to remove ipv6 lines AND those binded to all interfaces (marked in the output with asterisk *) in a single pass (in our setup, all IPv6 have IPv4 counterparts).

This script only lists ports between 16000 and 16255 (because thats my particular use case). You can remove the whole awk part to make it accept ANY port. Like this:

#!/bin/bash

IP_SCRIPT="/etc/openvpn/rsship"

PORTS=`lsof -i -P -n | grep LISTEN | grep sshtunnel | grep 127.0.0.1 | awk -F" " '{print $9}' | awk -F":" '{print $2}' | sort -n`


while IFS= read -r line; do
    REAL_IP=$($IP_SCRIPT $line)
    echo $line$'\t'$REAL_IP
done <<< "$PORTS"

The lsof part can be made better, though, it was already made, and since I was in a rush I decided to reuse that code without giving it a though. It is basically using awk to split and print a particular text of a column (the port, specifically). As we usually say: works on my machine. If you want to improve it, use "lsof -i -P -n | grep LISTEN" and see how can you narrow it down further it in a better way, as my line may not be suitable for your particular case. I'm not 100% sure, but I think that the same results can be achieved using only netstat.

If you're running this script very frequently and have multiple ports, you may consider giving it a twist so it calls netstat just once and process that output over and over to fetch all data.