The situation I'm in is where we have three separate interfaces on a particular machine. Each connected network only allows servers (by network-based routing/firewalling) to only offer particular services. Obviously, clients could be connecting from a wide variety of subnets and I'm relying on network firewalls for access controls on that.

The management interface allows ssh traffic, but in order for clients to be able to connect the default route must be set to the gateway on the management network. This resolves SSH but it breaks HTTP services which are only allowed over the Production interface. The third interface is a backup interface but all traffic for that one is on the same subnet.

The routing table looks something like this:

[root@xxxcpr2 ~]# ip route dev eth0 proto kernel scope link src dev eth1 proto kernel scope link src dev eth2 proto kernel scope link src
default via dev eth1

In the above, eth0 is the management interface, eth1 is the production interface, and eth2 is the backup interface.

Essentially, the condition I'm trying to get to is where the default gateway for any and all port 22 traffic is routed out the management interface but all other traffic outside a directly connected subnet should be routed out the production interface. Essentially, I'm 90% where I'm wanting to be, I just want to add an exception for SSH traffic.

I found this but I wasn't able to get it to work. Maybe it's my lack of understanding or some difference between RHEL and Ubuntu. Since the main table's default gateway was already set to go out the production interface, what I did was:

# echo 1 > /proc/sys/net/ipv4/ip_forward
# ip route add table 22 default via
# iptables -t mangle -A PREROUTING -p tcp --dport 22 -j MARK --set-mark 22
# ip rule add fwmark 22 table 22
# iptables -t nat -A POSTROUTING -o eth1 -j SNAT --to-source
# iptables -t nat -A POSTROUTING -o eth0 -j SNAT --to-source

To my way of thinking (maybe someone can point out how this is wrong) marking the packet and creating a rule to use table 22 should work (not sure why the original post mentioned SNAT but I tried to copy it as best I could for completeness) but connections still time out unless I change the main table's default route.

Basically, at this point I need someone to point out what I'm not understanding or that I'm overlooking.

  • It is not clear to me whether you are wanting to accept incoming ssh connections to your machine or whether you are trying to handle outgoing ssh connections to other machines. The rules for both are different. – Richm Nov 24 '14 at 18:43
  • Also are you using any connection tracking in your rules or is it just what you have posted above? – Richm Nov 24 '14 at 18:44
  • Just what I have above. Nice management is the only network that allows for ssh I was wanting both incoming and outgoing to use that interface. – Bratchley Nov 24 '14 at 19:05

For incoming ssh connections (without connection tracking)

iptables -t mangle -A OUTPUT -p tcp --dport 22 -j MARK --set-mark 22

For outgoing ssh connection (without connection tracking)

iptables -t mangle -A OUTPUT -p tcp --sport 22 -j MARK --set-mark 22

You should be able to combine them both using --port instead of --dport and --sport

It would likely be better practice to use connection tracking but the above rules should get you going.

| improve this answer | |
  • Are the nat rules required? – Bratchley Nov 24 '14 at 19:31
  • I added these rules but the packet counts don't ever go up after trying to connect over SSH. I added them to the OUTPUT chain as a test and the packet count for those rules went up like it was matching, but the traffic still failed. Is there a way to get it to log what routing table it's trying to use for the traffic? – Bratchley Nov 24 '14 at 19:49
  • I would leave the nat rules for now. They should be correct. – Richm Nov 24 '14 at 21:29
  • I tested the above (simple) rules and observed expected port 22 traffic going out of an alternative route. It worked both with and without the SNAT rules. – Richm Nov 26 '14 at 19:04

A more complete set of firewall rules that is based on a similar setup that I have for incoming ssh traffic.

Do make sure that any existing iptables rules are flushed before testing.

iptables -A INPUT   -m state --state ESTABLISHED,RELATED -j ACCEPT
iptables -A OUTPUT  -m state --state ESTABLISHED,RELATED -j ACCEPT
iptables -A FORWARD -m state --state ESTABLISHED,RELATED -j ACCEPT
iptables -A INPUT -p tcp -m tcp -m multiport --ports 22  -m state --state NEW  -j ACCEPT
iptables -A OUTPUT -p tcp -m tcp -m multiport --ports 22  -m state --state NEW  -j ACCEPT

iptables -t mangle -A PREROUTING -j CONNMARK --restore-mark
iptables -t mangle -A OUTPUT -j CONNMARK --restore-mark
iptables -N MarkThem -t mangle
iptables -t mangle -A OUTPUT -p tcp -m tcp -m multiport --ports 22  -j MarkThem
iptables -t mangle -A PREROUTING -p tcp -m tcp -m multiport --ports 22  -j MarkThem
# May need to filter out local LAN connections here.
# But try without filtering first.
# iptables -t mangle -A MarkThem  -s   -j RETURN
# iptables -t mangle -A MarkThem  -s   -j RETURN
# iptables -t mangle -A MarkThem  -s   -j RETURN
iptables -t mangle -A MarkThem  -j MARK --set-mark 22
iptables -t mangle -A MarkThem  -j CONNMARK --save-mark

iptables -t nat -A POSTROUTING -o eth1 -j SNAT --to-source
iptables -t nat -A POSTROUTING -o eth0 -j SNAT --to-source

Note I have explicitly set the rules to match both source or destination on port 22. You will likely want to tighten up the firewall rules once you have something working.

When testing outgoing ssh connections it might be worthwhile specifying a bind address with the -b option to see if it works for traffic that is sourced from your management address.

ssh -b auser@a.server.example.com

| improve this answer | |
  • The packet count is going up on the MarkThem chain but the SSH session is still timing out. – Bratchley Nov 25 '14 at 16:43
  • Use tcpdump to look at what is really going out on which interfaces. That should give you a clue as to what is going wrong. tcpdump -n -i eth0 port 22 – Richm Nov 26 '14 at 10:17
  • Here's the output of that command, it shows the client packets coming in eth0 but with no response going out. Earlier I did something similar with iptables -j LOG and it showed the response packets going out eth1 I tried changing the interface numbers in your command to show the same but it didn't show anything in tcpdump for some reason. – Bratchley Nov 26 '14 at 14:50
  • Hang on. In the commands I posted above I used --set-mark 1. Your routing rules are expecting --set-mark 22. Try changing that. I'll edit the answer. – Richm Nov 26 '14 at 16:52
  • Don't worry, I corrected for that when I first read it. – Bratchley Nov 26 '14 at 17:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.