2

I have two VPSs, first machine (proxy from now) is for proxy and second machine (dock from now) is docker host. I want to redirect all traffic generated inside a docker container itself over proxy, to not exposure dock machines public IP.

As connection between VPSs is over internet, no local connection, created a tunnel between them by ip tunnel as follows:

On proxy:

ip tunnel add tun10 mode ipip remote x.x.x.x local y.y.y.y dev eth0
ip addr add 192.168.10.1/24 peer 192.168.10.2 dev tun10
ip link set dev tun10 mtu 1492
ip link set dev tun10 up

On dock:

ip tunnel add tun10 mode ipip remote y.y.y.y local x.x.x.x dev eth0
ip addr add 192.168.10.2/24 peer 192.168.10.1 dev tun10
ip link set dev tun10 mtu 1492
ip link set dev tun10 up

PS: Do not know if ip tunnel can be used for production, it is another question, anyway planning to use libreswan or openvpn as a tunnel between VPSs.

After, added SNAT rules to iptables on both VPSs and some routing rules as follows:

On proxy:

iptables -t nat -A POSTROUTING -s 192.168.10.2/32 -j SNAT --to-source y.y.y.y

On dock:

iptables -t nat -A POSTROUTING -s 172.27.10.0/24 -j SNAT --to-source 192.168.10.2
ip route add default via 192.168.10.1 dev tun10 table rt2
ip rule add from 192.168.10.2 table rt2

And last but not least created a docker network with one test container attached to it as follows:

docker network create --attachable --opt com.docker.network.bridge.name=br-test --opt com.docker.network.bridge.enable_ip_masquerade=false --subnet=172.27.10.0/24 testnet
docker run --network testnet alpine:latest /bin/sh

Unfortunately all these ended with no success. So the question is how to debug that? Is it correct way? How would you do the redirection over proxy?

Some words about theory: Traffic coming from 172.27.10.0/24 subnet hits iptables SNAT rule, source IP changes to 192.168.10.2. By routing rule it routes over tun10 device, it is the tunnel. And hits another iptables SNAT rule that changes IP to y.y.y.y and finally goes to destination.

  • To check inside container you can use: apk add curl curl 'http://ipecho.net/plain' It should return y.y.y.y – teeper Jul 25 '18 at 6:07
1

I want to redirect all traffic generated inside a docker container itself over proxy, to not exposure dock machines public IP.

If that is all you want, you don't need NAT between the two docker instances.

You do need to enable forwarding on the proxy instance (e.g. via sysctl).

If the public IP of proxy is visible inside proxy, then you need to do SNAT with that public IP inside proxy only (not in dock). This is also called MASQUERADE. It's a standard setup, google for "masquerade".

If the public IP of proxy is not visible inside proxy (i.e., listed when you do ip addr), because the host of proxy (or even some other host further up) does the NAT then you you should avoid double NAT and make dock appear just like another container to the host. Details depend on how the network is setup on the host (which you didn't say), but basically the tunnel end in dock will have to have an IP in the same subnet as the containers on the proxy host.

An ipip tunnel is not encrypted; you should not use this tunnel unless the transport is entirely inside a trusted network (which is probably not the case, because you seem to want different public IP addresses). So use OpenVPN or easy to setup alternatives like tinc. Libreswan is not so easy to set up.

Edit

Step-by-step:

1) Verify the tunnel works. On proxy, do ping 192.168.10.2. On dock, do ping 192.168.10.1. Debug with tcpdump on all intervening network interfaces you have access to. If the ipip-tunnel doesn't work, use a different tunnel and make it work.

2) Remove all iptables rules on dock. Set the default route via the tunnel. Test with ip route get 8.8.8.8 if the route works.

3) Assuming that eth0 on proxy has the public IP, delete all iptables rules, do:

echo 1 > /proc/sys/net/ipv4/ip_forward
iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
iptables -A FORWARD -i eth0 -o tun10 -m state --state RELATED,ESTABLISHED -j ACCEPT
iptables -A FORWARD -o eth0 -i tun10 -j ACCEPT

This says "forward packets, masquerade all forwarded packets going out on eth0 by giving them the public IP address on eth0 as source address, and connection track them. Packets from tun10 can always be forwarded to eth0, incoming packets on eth0 only get forwarded to tun10 if the connection was established by dock."

Test by doing ping 8.8.8.8 on dock, while running tcpdump on tun10 and eth0 on proxy. You should see the packets forwarded, and the source being rewritten.

4) Once everything works, make it permanent: Edit /etc/sysctl.conf or a file in /etc/sysctl.d to have net.ipv4.ip_forward = 1. Either use a startup script to add the iptables rules, or use whatever package your distro provides to save the rules.

  • Thank you for reply. But as I see you did not understand correctly. There is no any docker instance in proxy, but docker instance is in dock machine. The thing that I want to achieve is to redirect the traffic from docker instance over proxy machine, not from dock machine itself, so while accessing some resource from inside instance your public IP will be the proxy public IP. Docker uses masquerading to provide connectivity out/in for its instances in a host. – teeper Jul 25 '18 at 7:26
  • It doesn't really matter if proxy and/or dock are docker instances or not If proxy is the host itself, the same applies: Is the public IP of proxy is visible in proxy (ip addr), or not (because your hosting service has other machines which do the NAT)? Depending on that, you either MASQUERADE only on proxy, or you pretend to whoever does the NAT that dock is just another machine, a peer of proxy, by forwarding/routing. – dirkt Jul 25 '18 at 9:02
  • You say: Details depend on how the network is setup on the host (which you didn't say)... Is not this enough? As connection between VPSs is over internet, no local connection,... – teeper Jul 25 '18 at 11:44
  • I cannot understand you. Can you please show me real working example. Thanks – teeper Jul 25 '18 at 11:45
  • Yes, I understood connection between VPSs is over the internet. This is not the point. Can you please look at the output of ip addr on proxy, and see if the public IP is listed there, or not? And if it is not listed, could you describe how proxy connects to whoever supplies the public IP? That is the point. – dirkt Jul 25 '18 at 11:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.