At work we have and end-to-end system that we will route packets through to pop out the other side, and then travel back through to come back where they originated. You can think of this as a normal communications stack.

In order to test throughput, we find ourselves using iperf3 by using two external computers to send traffic to this stack. We find ourselves using three computers total because we haven't been able to solve this route problem, but I feel that there must be some solution we're not seeing as I'm sure this has come up before. Id like to use just one machine if possible.

Let's say our environment looks like this:enter image description here

Where A is an iperf3 client, B consists of multiple programs passing packets off to each other as ethernet packets (Kept as one black box for simplicity) and C is the iperf3 server.

So to recap, I'd like packets to travel from A, to B, to C, and then back to B, and back to A.

enter image description here

To do this using Iperf, I'd have a client at A sending packets to C.

iperf3 -c "C"

And I'd also set up a server at C using

iperf3 -s

However, without any routing rules at this point these packets will bypass B and go straight between A and C. To stop this from happening, I make a routing rule to say something like "Route all traffic destined to C to B instead".

route add "C" netmask gw "B"

Also, we have to do the same thing on the other side so that the server will not send directly back to A.

route add "A" netmask gw "B"

If you haven't caught it already, this will cause a paradox and cause all traffic destined for both A and C to B. This means when B is done processing and would like to send the data to C, it will instead send to itself because this is what the routing table dictates*. This will also happen in the opposite direction.

Does anyone know a way around this? I'm not sure if it can be done with simple routing, but I don't know much about routing to begin with so maybe there's a trick for this?

*I'm not actually sure if it will keep sending to itself (I don't think it will, even if it does it won't know what to do with it the second time) but this is me trying to say it will not work with this configuration.

  • Not sure if I follow the description. You have three computers A, B and C connected to the same LAN. If you set a special route on A and a different special route on C, I don't see why B should be affected - there is no "paradox" because A, B and C have different routing rules. If I misunderstood your configuration, please update question. – dirkt Oct 20 '17 at 5:48
  • Your example misses the LAN connections, the switch/routers and networks IP address examples etc. Without those we can assume that they are all connected point-to-point to each other? But then you wouldn't have that issue ;). – Ziazis Oct 20 '17 at 7:18
  • @dirkt I somehow forgot to mention this is all on the same computer, I'll add it now – blurb Oct 20 '17 at 11:18
  • Ah, so the goal is to do this on one computer, but currently you have to use three. Simplest way: add two network namespaces, one for A and one for C, with corresponding veth pairs. Then you have three "virtual computers", each with their own routing table. – dirkt Oct 20 '17 at 11:22
  • @dirkt yes exactly, I will look in to namespaces. – blurb Oct 20 '17 at 11:24

This almost sounds like something you could achieve with iptables. Something like this could be put on "B":

iptables -t nat -A PREROUTING -s "A" -p tcp --dport 5201 -j DNAT --to-destination C:5201

iptables -t nat -A PREROUTING -s "C" -p tcp --dport 5201 -j DNAT --to-destination A:5201

Sounds like you need a (transparent) proxy, try to look into that.

However if you actually want to use routing since only those two clients should be affected you'd have to configure some kind of routing functionality on B first (or some kind of ip forwarding at least). Then you'd need to add the route on the hosts which would be something like this.

On A

route add C netmask gw B

On C

route add A netmask gw B

E.g. you'd tell A that to reach C the next hop for that would not be your default gateway but your B box. The 255 in all four octs is required and means only this IP. So it doesn't mean all traffic that has the destination C rather go to B but it means all traffic ment to go to C - instead of the default gw - use B as your gw. So there is no paradox unless you haven't configured anything on the B box. Then of course the B box will just drop the packages and probably look at you "why the hell are you sending me this ;)".

Another alternative is, that you mirror a port of one of your hosts, if you have a managed switch, so that every package sent or received on that one LAN port is also delivered to your B box on that mirrored port.

There are alot of other alternatives to cut into the communication between those two clients with your third client. However with the description you gave you don't really have any requirements. So try anything.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.