Timeout when connecting RDS using AWS session manager via EC2 instance

Question

I am having trouble accessing AWS RDS instance in a private subnet using session manager through ec2 instance (Ubuntu).

Port forwarding using AWS Session manager -

    aws ssm start-session --target $INSTANCE_ID \
                       --document-name AWS-StartPortForwardingSession \
                       --parameters '{"portNumber":[3307],"localPortNumber":["8000"]}' \
                                             --profile=production \
                       --region=us-east-1`

Now local port 8000 successfully forwarded to port 3307 of EC2.

lsof -nP +c 15 | grep LISTEN

session-manager 59744 xrage 12u IPv4 0xa6c821d0a800e9a1 0t0 TCP 127.0.0.1:8000 (LISTEN)

EC2 instance has an iptable rule that forward all traffic on port 3307 to rds host at port 3306 (MYSQL PORT).

sudo iptables -t nat -A OUTPUT -o lo -d 127.0.0.1 -p tcp --dport 3307 -j DNAT  --to-destination <rds_private_ip>:3306

This allowed me to access remote host on the desired port from ec2, But still, the remote host is not accessible locally on port 8000.

Network details on the local machine -

• Not using any proxy
• Firewall disabled on mac

So it's clear that SSM setup is working but something else is blocking this traffic, Need some help here.

Answer 1

When you said that remote host can access MySQL instance, u tried that port (telnet) or mysql command line client, just to confirm?

Assuming yes to the above, I would move on to the next step. It is my understanding that AWS-StartPortForwardingSession is to cover this scenario: You have something running on your EC2, say MySQL db that you hand rolled and installed. You want to forward traffic from your workstation (for example) to the MySQL instance running on EC2, via SSM. That's what AWS-StartPortForwardingSession was designed for.

In your case, the EC2 is more of a bastion host, and your MySQL instance is RDS type. So your EC2 does not host anything -- it simply is a gateway to connect to your RDS.

Assuming the above assumption is correct, I would use AWS-StartSSHSession instead (https://docs.aws.amazon.com/systems-manager/latest/userguide/session-manager-getting-started-enable-ssh-connections.html).

The full answer would be verbose, but I just did it a few months ago: https://medium.com/@clearwaterstream/recipe-connect-to-an-rds-database-in-a-private-subnet-from-your-workstation-over-https-c66db6ead9f0