2

I am using a debian based gateway for my network. It has four interfaces:

  • eth0: dynamic (connected to ISP)
  • eth1:
    • ip 192.168.1.1
    • connected to a switch
    • dnsmasq assigns ip addresses to connected clients (192.168.1.*)
  • eth2:
    • ip 192.168.2.1
    • connected to a switch
    • dnsmasq assigns ip addresses to connected clients (192.168.2.*)
  • wlan0: static
    • ip 192.168.3.1
    • acting as an AP
    • dnsmasq assigns ip addresses to connected clients (192.168.3.*)

Clients connected through either interface are able to access the internet and are able to access services on the gateway:

  • ping 192.168.1.1 from 192.168.3.111 works
  • ping 192.168.3.1 from 192.168.1.110 works

Clients are not able to contact clients on other subnets:

  • ping 192.168.1.40 from 192.168.3.111 does not work

ip route shows the following:

default via 80.0.0.1 dev eth0 
80.0.0.0/24 dev eth0 proto kernel scope link src 80.0.0.7 
192.168.1.0/24 dev eth1 proto kernel scope link src 192.168.1.1 
192.168.2.0/24 dev eth2 proto kernel scope link src 192.168.2.1 linkdown 
192.168.3.0/24 dev wlan0 proto kernel scope link src 192.168.3.1 

iptables is not a problem

cat /proc/sys/net/ipv4/ip_forward returns 1

Question: How do I achieve that all clients are able to contact each other without adjusting the network settings of every client?


Added information: iptables-save -c:

# Generated by iptables-save v1.6.0 on Sat Feb  2 19:03:01 2019
*mangle
:PREROUTING ACCEPT [7068132:2249036546]
:INPUT ACCEPT [6634829:1954826260]
:FORWARD ACCEPT [432992:294164216]
:OUTPUT ACCEPT [3915469:40516939510]
:POSTROUTING ACCEPT [4348507:40811115290]
COMMIT
# Completed on Sat Feb  2 19:03:01 2019
# Generated by iptables-save v1.6.0 on Sat Feb  2 19:03:01 2019
*nat
:PREROUTING ACCEPT [3681:370156]
:INPUT ACCEPT [1605:106410]
:OUTPUT ACCEPT [6748:465680]
:POSTROUTING ACCEPT [325:26525]
[171:12116] -A POSTROUTING -o eth0 -j MASQUERADE
[7374:521015] -A POSTROUTING -o tun0 -j MASQUERADE
COMMIT
# Completed on Sat Feb  2 19:03:01 2019
# Generated by iptables-save v1.6.0 on Sat Feb  2 19:03:01 2019
*filter
:INPUT DROP [219:37927]
:FORWARD DROP [0:0]
:OUTPUT DROP [0:0]
:garbage - [0:0]
[4503:318377] -A INPUT -i lo -j ACCEPT
[3186066:177042527] -A INPUT -i eth1 -j ACCEPT
[16:3840] -A INPUT -i eth2 -j ACCEPT
[523:97073] -A INPUT -i wlan0 -j ACCEPT
[2130075:1081232616] -A INPUT -i eth0 -m state --state RELATED,ESTABLISHED -j ACCEPT
[1312840:695988466] -A INPUT -i tun0 -m state --state RELATED,ESTABLISHED -j ACCEPT
[0:0] -A INPUT -d 192.168.3.130/32 -i eth0 -j REJECT --reject-with icmp-port-unreachable
[798:139319] -A INPUT -j garbage
[5797:6921736] -A FORWARD -i eth0 -o eth1 -m state --state RELATED,ESTABLISHED -j ACCEPT
[5902:682788] -A FORWARD -i eth1 -o eth0 -j ACCEPT
[0:0] -A FORWARD -i eth0 -o eth2 -m state --state RELATED,ESTABLISHED -j ACCEPT
[0:0] -A FORWARD -i eth2 -o eth0 -j ACCEPT
[0:0] -A FORWARD -i eth0 -o wlan0 -m state --state RELATED,ESTABLISHED -j ACCEPT
[0:0] -A FORWARD -i wlan0 -o eth0 -j ACCEPT
[0:0] -A FORWARD -i tun0 -o eth0 -m state --state RELATED,ESTABLISHED -j ACCEPT
[0:0] -A FORWARD -i eth0 -o tun0 -j ACCEPT
[0:0] -A FORWARD -i eth1 -o eth2 -j ACCEPT
[0:0] -A FORWARD -i eth2 -o eth1 -j ACCEPT
[42490:6707236] -A FORWARD -i eth1 -o wlan0 -j ACCEPT
[42582:6765739] -A FORWARD -i wlan0 -o eth1 -j ACCEPT
[224512:260134799] -A FORWARD -i tun0 -o eth1 -m state --state RELATED,ESTABLISHED -j ACCEPT
[97009:7301815] -A FORWARD -i eth1 -o tun0 -j ACCEPT
[0:0] -A FORWARD -i tun0 -o eth2 -m state --state RELATED,ESTABLISHED -j ACCEPT
[0:0] -A FORWARD -i eth2 -o tun0 -j ACCEPT
[0:0] -A FORWARD -i eth2 -o wlan0 -j ACCEPT
[0:0] -A FORWARD -i wlan0 -o eth2 -j ACCEPT
[7517:1453488] -A FORWARD -i wlan0 -o tun0 -j ACCEPT
[7178:4193761] -A FORWARD -i tun0 -o wlan0 -m state --state RELATED,ESTABLISHED -j ACCEPT
[0:0] -A FORWARD -s 192.168.10.0/24 -m policy --dir in --pol ipsec --proto esp -j ACCEPT
[0:0] -A FORWARD -s 192.168.10.0/24 -m policy --dir out --pol ipsec --proto esp -j ACCEPT
[0:0] -A FORWARD -p tcp -m tcp --tcp-flags FIN,SYN,RST,ACK SYN -m limit --limit 1/sec -j ACCEPT
[0:0] -A FORWARD -p tcp -m tcp --tcp-flags FIN,SYN,RST,ACK RST -m limit --limit 1/sec -j ACCEPT
[0:0] -A FORWARD -p icmp -m icmp --icmp-type 8 -m limit --limit 1/sec -j ACCEPT
[0:0] -A FORWARD -j garbage
[0:0] -A OUTPUT -d 192.168.0.0/16 -o wlan0 -p tcp -m tcp --sport 9091 -m owner --uid-owner 109 -j ACCEPT
[0:0] -A OUTPUT -d 192.168.0.0/16 -o wlan0 -p udp -m udp --sport 9091 -m owner --uid-owner 109 -j ACCEPT
[2131:982946] -A OUTPUT -d 192.168.0.0/16 -o eth1 -p tcp -m tcp --sport 9091 -m owner --uid-owner 109 -j ACCEPT
[0:0] -A OUTPUT -d 192.168.0.0/16 -o eth1 -p udp -m udp --sport 9091 -m owner --uid-owner 109 -j ACCEPT
[0:0] -A OUTPUT -d 192.168.0.0/16 -o eth2 -p tcp -m tcp --sport 9091 -m owner --uid-owner 109 -j ACCEPT
[0:0] -A OUTPUT -d 192.168.0.0/16 -o eth2 -p udp -m udp --sport 9091 -m owner --uid-owner 109 -j ACCEPT
[1026585:1247589085] -A OUTPUT -o tun0 -m owner --uid-owner 109 -j ACCEPT
[218:15112] -A OUTPUT -o lo -m owner --uid-owner 109 -j ACCEPT
[0:0] -A OUTPUT -m owner --uid-owner 109 -j REJECT --reject-with icmp-port-unreachable
[4285:303265] -A OUTPUT -o lo -j ACCEPT
[643670:37873467432] -A OUTPUT -o eth1 -j ACCEPT
[16:3840] -A OUTPUT -o eth2 -j ACCEPT
[479:110432] -A OUTPUT -o wlan0 -j ACCEPT
[2230895:1393337874] -A OUTPUT -o eth0 -j ACCEPT
[7182:1129654] -A OUTPUT -o tun0 -j ACCEPT
[0:0] -A OUTPUT -j garbage
[0:0] -A garbage -p icmp -j LOG --log-prefix "DROP ICMP-Packet: "
[355:109444] -A garbage -p udp -j LOG --log-prefix "DROP UDP-Packet: "
[1022:131267] -A garbage -p tcp -j LOG --log-prefix "DROP TCP-Packet: "
COMMIT
# Completed on Sat Feb  2 19:03:01 2019

Activated tcpdump while:

  • ping 192.168.1.110 from 192.168.3.130 (100% packet loss)
  • ping 192.168.3.140 from 192.168.1.110 (0.0% packet loss)
  • accessing 192.168.1.40:5000 from 192.168.3.140 (no response)

Summary from tcpdump:

77136 packets captured
77363 packets received by filter
0 packets dropped by kernel


Analysis of tcpdump (filtered ICMP entries)

ping from 192.168.3.130 to 192.168.1.40:

14:33:10.404428 IP pizerow > 192.168.1.40: ICMP echo request, id 15599, seq 1, length 64
14:33:11.443861 IP pizerow > 192.168.1.40: ICMP echo request, id 15599, seq 2, length 64
14:33:12.483868 IP pizerow > 192.168.1.40: ICMP echo request, id 15599, seq 3, length 64
14:33:13.523863 IP pizerow > 192.168.1.40: ICMP echo request, id 15599, seq 4, length 64
14:33:14.563859 IP pizerow > 192.168.1.40: ICMP echo request, id 15599, seq 5, length 64
14:33:15.603854 IP pizerow > 192.168.1.40: ICMP echo request, id 15599, seq 6, length 64
14:33:16.643855 IP pizerow > 192.168.1.40: ICMP echo request, id 15599, seq 7, length 64
14:33:17.683844 IP pizerow > 192.168.1.40: ICMP echo request, id 15599, seq 8, length 64
14:33:18.723842 IP pizerow > 192.168.1.40: ICMP echo request, id 15599, seq 9, length 64
14:33:19.763853 IP pizerow > 192.168.1.40: ICMP echo request, id 15599, seq 10, length 64

Corresponding entries on wlan0 (interface for the 3-subnet) at the gateway:

14:33:10.506374 IP pizerow > 192.168.1.40: ICMP echo request, id 15599, seq 1, length 64
14:33:11.549063 IP pizerow > 192.168.1.40: ICMP echo request, id 15599, seq 2, length 64
14:33:12.589103 IP pizerow > 192.168.1.40: ICMP echo request, id 15599, seq 3, length 64
14:33:13.629124 IP pizerow > 192.168.1.40: ICMP echo request, id 15599, seq 4, length 64
14:33:14.669151 IP pizerow > 192.168.1.40: ICMP echo request, id 15599, seq 5, length 64
14:33:15.709198 IP pizerow > 192.168.1.40: ICMP echo request, id 15599, seq 6, length 64
14:33:16.749188 IP pizerow > 192.168.1.40: ICMP echo request, id 15599, seq 7, length 64
14:33:17.789169 IP pizerow > 192.168.1.40: ICMP echo request, id 15599, seq 8, length 64
14:33:18.829243 IP pizerow > 192.168.1.40: ICMP echo request, id 15599, seq 9, length 64

Corresponding entries on eth1(interface for the 1-subnet) at the gateway:

14:33:10.506430 IP pizerow > 192.168.1.40: ICMP echo request, id 15599, seq 1, length 64
14:33:10.506703 IP 192.168.1.40 > pizerow: ICMP echo reply, id 15599, seq 1, length 64
14:33:11.549119 IP pizerow > 192.168.1.40: ICMP echo request, id 15599, seq 2, length 64
14:33:11.549373 IP 192.168.1.40 > pizerow: ICMP echo reply, id 15599, seq 2, length 64
14:33:12.589157 IP pizerow > 192.168.1.40: ICMP echo request, id 15599, seq 3, length 64
14:33:12.589431 IP 192.168.1.40 > pizerow: ICMP echo reply, id 15599, seq 3, length 64
14:33:13.629182 IP pizerow > 192.168.1.40: ICMP echo request, id 15599, seq 4, length 64
14:33:13.629458 IP 192.168.1.40 > pizerow: ICMP echo reply, id 15599, seq 4, length 64
14:33:14.669207 IP pizerow > 192.168.1.40: ICMP echo request, id 15599, seq 5, length 64
14:33:14.669486 IP 192.168.1.40 > pizerow: ICMP echo reply, id 15599, seq 5, length 64
14:33:15.709273 IP pizerow > 192.168.1.40: ICMP echo request, id 15599, seq 6, length 64
14:33:15.709547 IP 192.168.1.40 > pizerow: ICMP echo reply, id 15599, seq 6, length 64
14:33:16.749244 IP pizerow > 192.168.1.40: ICMP echo request, id 15599, seq 7, length 64
14:33:16.749522 IP 192.168.1.40 > pizerow: ICMP echo reply, id 15599, seq 7, length 64
14:33:17.789224 IP pizerow > 192.168.1.40: ICMP echo request, id 15599, seq 8, length 64
14:33:17.789496 IP 192.168.1.40 > pizerow: ICMP echo reply, id 15599, seq 8, length 64
14:33:18.829295 IP pizerow > 192.168.1.40: ICMP echo request, id 15599, seq 9, length 64
14:33:18.829574 IP 192.168.1.40 > pizerow: ICMP echo reply, id 15599, seq 9, length 64
14:33:19.869300 IP pizerow > 192.168.1.40: ICMP echo request, id 15599, seq 10, length 64
14:33:19.869576 IP 192.168.1.40 > pizerow: ICMP echo reply, id 15599, seq 10, length 64

tcpdump without iptables-firewall

Ran the following:

#!/bin/sh
iptables=/sbin/iptables
$iptables -F
$iptables -X
$iptables -Z
$iptables -t nat -F
$iptables -t nat -X
$iptables -t filter -F
$iptables -t mangle -F
$iptables -t mangle -X
echo 1 > /proc/sys/net/ipv4/ip_forward
$iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
$iptables -t nat -A POSTROUTING -o tun0 -j MASQUERADE
$iptables -P INPUT ACCEPT
$iptables -P OUTPUT ACCEPT
$iptables -P FORWARD ACCEPT

ping -c 5 192.168.1.40 from 192.168.3.130: 100% packet loss

tcpdump on interface wlan0 on gateway:

16:04:15.653830 IP pizerow > 192.168.1.40: ICMP echo request, id 16003, seq 1, length 64
16:04:16.663534 IP pizerow > 192.168.1.40: ICMP echo request, id 16003, seq 2, length 64
16:04:17.705299 IP pizerow > 192.168.1.40: ICMP echo request, id 16003, seq 3, length 64
16:04:18.743570 IP pizerow > 192.168.1.40: ICMP echo request, id 16003, seq 4, length 64
16:04:19.783548 IP pizerow > 192.168.1.40: ICMP echo request, id 16003, seq 5, length 64

tcpdump on interface eth1 on gateway:

16:04:15.653895 IP pizerow > 192.168.1.40: ICMP echo request, id 16003, seq 1, length 64
16:04:15.654178 IP 192.168.1.40 > pizerow: ICMP echo reply, id 16003, seq 1, length 64
16:04:16.663579 IP pizerow > 192.168.1.40: ICMP echo request, id 16003, seq 2, length 64
16:04:16.663848 IP 192.168.1.40 > pizerow: ICMP echo reply, id 16003, seq 2, length 64
16:04:17.705391 IP pizerow > 192.168.1.40: ICMP echo request, id 16003, seq 3, length 64
16:04:17.705676 IP 192.168.1.40 > pizerow: ICMP echo reply, id 16003, seq 3, length 64
16:04:18.743631 IP pizerow > 192.168.1.40: ICMP echo request, id 16003, seq 4, length 64
16:04:18.743907 IP 192.168.1.40 > pizerow: ICMP echo reply, id 16003, seq 4, length 64
16:04:19.783596 IP pizerow > 192.168.1.40: ICMP echo request, id 16003, seq 5, length 64
  • 1
    It has to be related to your NAT rules which probably NAT each net between each others (and there's an explaination for your .1 addresses reachable too), so iptables is probably the problem. Please add the output of this in your question: iptables-save -c. Once provided it will be easy to tell how to correct them – A.B Feb 2 at 17:51
  • 1
    Your iptables rules are pretty complicated, so I'm curious why you can confidently state they are not a problem. Ways to debug this: (1) Delete all iptables rules, make sure forwarding ACCEPTs. If that works, you found the problem. (2) Run tcpdump on all four involved interfaces while ping 192.168.1.40 from 192.168.3.111, see if the problem is really forwarding on the gateway, or if there's another point of failure. – dirkt Feb 2 at 18:38
  • 1
    And if there's a mistake in the table that drops without jumping to garbage to log, you won't see any logs entries.. – dirkt Feb 2 at 19:52
  • 2
    Ok I admit I'm a bt baffled. NAT rules appear fine, it's the only simple part heh. Are you sure some clients aren't firewalled themselves, – A.B Feb 2 at 20:36
  • 1
    So now you verified the iptables rules on the router are not the cause, next do (2) on all four interfaces (two on router, one each on end hosts). Depending on how far the ping gets, you'll see where the problem is. If you need to check effective routing (on either of the three machines), look at ip route get .... – dirkt Feb 3 at 10:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.