trouble setting proper forwarding rules in `iptables` with custom ip address for network sharing

Question

I am having issues setting up a bridge for my raspberry pi.

My setup is:
I have a laptop running fedora 27 workstation which is connected to the internet over wifi. I have a Raspberry Pi Zero W which is connected to my laptop via usb (and only usb, no external power, no ethernet, nothing).
I flashed stretch lite image to my pi and then installed P4wnP1 from here: https://github.com/mame82/P4wnP1
Before i installed P4wnP1 my pi had a random 169.254.xxx.xxx address, which is why i changed the ip of my usb ethernet interface to a proper subnet to ssh into the pi. After a while i figured out the right setup to get my pi online and download git to clone the repo.

After i ran the install.sh and rebooted the pi the pi had a static ip address 172.16.0.1. And i tried the same thing to get it online, changed the ip of my interface, ssh to the pi, set up the gateway to my fedora machine.
But i cannot get the pi online.

I should probably mention here that i enabled "share connection to other computers" in network manager and also tried a lot of things with iptables, but i cannot get it to work.

I have spent the past 3 days trying to figure it out, but i had no success.

here is my ifconfig on my fedora:

$ ifconfig
enp0s20f0u6i1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 172.16.0.2  netmask 255.255.0.0  broadcast 172.16.255.255
        inet6 fe80::f7f7:80c:8a15:5771  prefixlen 64  scopeid 0x20<link>
        ether ee:98:9b:bc:37:ab  txqueuelen 1000  (Ethernet)
        RX packets 2687  bytes 186674 (182.2 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 1648  bytes 176862 (172.7 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

enp0s31f6: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        ether c8:5b:76:6b:e4:90  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        device interrupt 16  memory 0xf1200000-f1220000  

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 1982  bytes 177290 (173.1 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 1982  bytes 177290 (173.1 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

virbr0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 192.168.122.1  netmask 255.255.255.0  broadcast 192.168.122.255
        ether 52:54:00:08:e4:d3  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

wlp4s0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.2.106  netmask 255.255.255.0  broadcast 192.168.2.255
        inet6 fe80::ebcf:d3b1:5a74:185e  prefixlen 64  scopeid 0x20<link>
        ether e4:a7:a0:99:2e:8d  txqueuelen 1000  (Ethernet)
        RX packets 135496  bytes 72791497 (69.4 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 51579  bytes 21450089 (20.4 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

here the enp0s20f06i3 interface is the one connected to the pi. Before i changed its ip address it had a 10.46.0.1 address, which is also the same address after reboot.

here route -n from my pi

pi@MAME82-P4WNP1:~ $ route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         172.16.0.2      0.0.0.0         UG    0      0        0 usb0
172.16.0.0      0.0.0.0         255.255.255.252 U     0      0        0 usb0
172.24.0.0      0.0.0.0         255.255.255.0   U     0      0        0 wlan0

and the ifconfig of my pi

pi@MAME82-P4WNP1:~ $ ifconfig
lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1  (Local Loopback)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

usb0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 172.16.0.1  netmask 255.255.255.252  broadcast 172.16.0.3
        inet6 fe80::cc4b:62ff:fe84:7df0  prefixlen 64  scopeid 0x20<link>
        ether ce:4b:62:84:7d:f0  txqueuelen 1000  (Ethernet)
        RX packets 1959  bytes 182340 (178.0 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 3197  bytes 269463 (263.1 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

wlan0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 172.24.0.1  netmask 255.255.255.0  broadcast 172.24.0.255
        inet6 fe80::ba27:ebff:fe5e:ceb7  prefixlen 64  scopeid 0x20<link>
        ether b8:27:eb:5e:ce:b7  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 14  bytes 1404 (1.3 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

and here route -n on my fedora

$ route -n                 
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.2.1     0.0.0.0         UG    600    0        0 wlp4s0
172.16.0.0      0.0.0.0         255.255.0.0     U     0      0        0 enp0s20f0u6i1
192.168.2.0     0.0.0.0         255.255.255.0   U     600    0        0 wlp4s0
192.168.122.0   0.0.0.0         255.255.255.0   U     0      0        0 virbr0

resolv.conf on my pi

pi@MAME82-P4WNP1:~ $ cat /etc/resolv.conf
# Generated by resolvconf
nameserver 10.46.0.1
nameserver 8.8.8.8
nameserver 8.8.4.4

and /etc/network/interfaces on my pi

pi@MAME82-P4WNP1:~ $ cat /etc/network/interfaces
# interfaces(5) file used by ifup(8) and ifdown(8)

# Please note that this file is written to be used with dhcpcd
# For static IP, consult /etc/dhcpcd.conf and 'man dhcpcd.conf'

# Include files from /etc/network/interfaces.d:
source-directory /etc/network/interfaces.d


dns-nameservers 8.8.8.8 8.8.4.4

auto usb0

iface usb0 inet manual

auto usb1

iface usb1 inet manual

finally my iptables on my fedora, where i think the issue is:

$ sudo iptables -L                                                          
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         
ACCEPT     udp  --  anywhere             anywhere             udp dpt:bootps
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:bootps
ACCEPT     udp  --  anywhere             anywhere             udp dpt:domain
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:domain
ACCEPT     udp  --  anywhere             anywhere             udp dpt:domain
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:domain
ACCEPT     udp  --  anywhere             anywhere             udp dpt:bootps
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:bootps
ACCEPT     all  --  anywhere             anywhere             ctstate RELATED,ESTABLISHED
ACCEPT     all  --  anywhere             anywhere            
INPUT_direct  all  --  anywhere             anywhere            
INPUT_ZONES_SOURCE  all  --  anywhere             anywhere            
INPUT_ZONES  all  --  anywhere             anywhere            
DROP       all  --  anywhere             anywhere             ctstate INVALID
REJECT     all  --  anywhere             anywhere             reject-with icmp-host-prohibited

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         
ACCEPT     all  --  anywhere             10.42.0.0/24         state RELATED,ESTABLISHED
ACCEPT     all  --  10.42.0.0/24         anywhere            
ACCEPT     all  --  anywhere             anywhere            
REJECT     all  --  anywhere             anywhere             reject-with icmp-port-unreachable
REJECT     all  --  anywhere             anywhere             reject-with icmp-port-unreachable
ACCEPT     all  --  anywhere             192.168.122.0/24     ctstate RELATED,ESTABLISHED
ACCEPT     all  --  192.168.122.0/24     anywhere            
ACCEPT     all  --  anywhere             anywhere            
REJECT     all  --  anywhere             anywhere             reject-with icmp-port-unreachable
REJECT     all  --  anywhere             anywhere             reject-with icmp-port-unreachable
ACCEPT     all  --  anywhere             anywhere             ctstate RELATED,ESTABLISHED
ACCEPT     all  --  anywhere             anywhere            
FORWARD_direct  all  --  anywhere             anywhere            
FORWARD_IN_ZONES_SOURCE  all  --  anywhere             anywhere            
FORWARD_IN_ZONES  all  --  anywhere             anywhere            
FORWARD_OUT_ZONES_SOURCE  all  --  anywhere             anywhere            
FORWARD_OUT_ZONES  all  --  anywhere             anywhere            
DROP       all  --  anywhere             anywhere             ctstate INVALID
REJECT     all  --  anywhere             anywhere             reject-with icmp-host-prohibited
ACCEPT     all  --  anywhere             anywhere            
ACCEPT     all  --  anywhere             anywhere            
ACCEPT     all  --  anywhere             anywhere            

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         
ACCEPT     udp  --  anywhere             anywhere             udp dpt:bootpc
OUTPUT_direct  all  --  anywhere             anywhere            

Chain FORWARD_IN_ZONES (1 references)
target     prot opt source               destination         
FWDI_FedoraWorkstation  all  --  anywhere             anywhere            [goto] 
FWDI_FedoraWorkstation  all  --  anywhere             anywhere            [goto] 
FWDI_FedoraWorkstation  all  --  anywhere             anywhere            [goto] 

Chain FORWARD_IN_ZONES_SOURCE (1 references)
target     prot opt source               destination         

Chain FORWARD_OUT_ZONES (1 references)
target     prot opt source               destination         
FWDO_FedoraWorkstation  all  --  anywhere             anywhere            [goto] 
FWDO_FedoraWorkstation  all  --  anywhere             anywhere            [goto] 
FWDO_FedoraWorkstation  all  --  anywhere             anywhere            [goto] 

Chain FORWARD_OUT_ZONES_SOURCE (1 references)
target     prot opt source               destination         

Chain FORWARD_direct (1 references)
target     prot opt source               destination         

Chain FWDI_FedoraWorkstation (3 references)
target     prot opt source               destination         
FWDI_FedoraWorkstation_log  all  --  anywhere             anywhere            
FWDI_FedoraWorkstation_deny  all  --  anywhere             anywhere            
FWDI_FedoraWorkstation_allow  all  --  anywhere             anywhere            
ACCEPT     icmp --  anywhere             anywhere            

Chain FWDI_FedoraWorkstation_allow (1 references)
target     prot opt source               destination         

Chain FWDI_FedoraWorkstation_deny (1 references)
target     prot opt source               destination         

Chain FWDI_FedoraWorkstation_log (1 references)
target     prot opt source               destination         

Chain FWDO_FedoraWorkstation (3 references)
target     prot opt source               destination         
FWDO_FedoraWorkstation_log  all  --  anywhere             anywhere            
FWDO_FedoraWorkstation_deny  all  --  anywhere             anywhere            
FWDO_FedoraWorkstation_allow  all  --  anywhere             anywhere            

Chain FWDO_FedoraWorkstation_allow (1 references)
target     prot opt source               destination         

Chain FWDO_FedoraWorkstation_deny (1 references)
target     prot opt source               destination         

Chain FWDO_FedoraWorkstation_log (1 references)
target     prot opt source               destination         

Chain INPUT_ZONES (1 references)
target     prot opt source               destination         
IN_FedoraWorkstation  all  --  anywhere             anywhere            [goto] 
IN_FedoraWorkstation  all  --  anywhere             anywhere            [goto] 
IN_FedoraWorkstation  all  --  anywhere             anywhere            [goto] 

Chain INPUT_ZONES_SOURCE (1 references)
target     prot opt source               destination         

Chain INPUT_direct (1 references)
target     prot opt source               destination         

Chain IN_FedoraWorkstation (3 references)
target     prot opt source               destination         
IN_FedoraWorkstation_log  all  --  anywhere             anywhere            
IN_FedoraWorkstation_deny  all  --  anywhere             anywhere            
IN_FedoraWorkstation_allow  all  --  anywhere             anywhere            
ACCEPT     icmp --  anywhere             anywhere            

Chain IN_FedoraWorkstation_allow (1 references)
target     prot opt source               destination         
ACCEPT     udp  --  anywhere             anywhere             udp dpt:netbios-ns ctstate NEW
ACCEPT     udp  --  anywhere             anywhere             udp dpt:netbios-dgm ctstate NEW
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:ssh ctstate NEW
ACCEPT     udp  --  anywhere             224.0.0.251          udp dpt:mdns ctstate NEW
ACCEPT     udp  --  anywhere             anywhere             udp dpts:blackjack:65535 ctstate NEW
ACCEPT     tcp  --  anywhere             anywhere             tcp dpts:blackjack:65535 ctstate NEW

Chain IN_FedoraWorkstation_deny (1 references)
target     prot opt source               destination         

Chain IN_FedoraWorkstation_log (1 references)
target     prot opt source               destination         

Chain OUTPUT_direct (1 references)
target     prot opt source               destination        

I think i need to just add the proper entries, but i could not figure it out, i searched a lot of forums.
is there a way to change the 10.46.0.0/24 entries to the 172.16.0.0/24 network?
because my interface had that ip before and if i could just swap the ip in the rules i would be done, right?

I tried sudo iptables -t nat -A POSTROUTING -o wlp4s0 -j MASQUERADE and also tried to set the rules myself, however i cannot manage to set my FORWARD rules accordingly.

Answer 1

Summary:

Pi needs Fedora to forward traffic to the internet.

Pi

1 network card (that we care about) named:

usb0 -- connected to Fedora.

Fedora:

Internet connected.

2 network cards (that we care about) named:

wlp4s0 -- wifi internet

enp0s20f0u6i1 -- connected to the pi.

To make life simpler I recommend stopping the Predictable Network Interface Names thingy. We want to use nic names and do not want them to change on us.

Step 1: Stop systemd's Predictable Network Interface Names thingy by adding "net.ifnames=0" to kernel command line.

sudo vi /etc/default/grub

GRUB_CMDLINE_LINUX="net.ifnames=0"

Now update grub:

sudo grub-mkconfig -o /boot/grub/grub.cfg

note:

I have seen where the value "biosdevname=0" was added to the kernal command line in addition to net.ifnames=0. My setup did not require it.

Step 2:

Assign a new name using udev rules by creating a new rule file

sudo vi /etc/udev/rules.d/10-myCustom-net.rules

SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="08:00:27:f3:79:59", KERNEL=="eth*", NAME="test0"

MUST change the ATTR{address}=="08:00:27:f3:79:59" line to your MAC address.

Change NAME="test0" to the name you want to give the nic.

note:

Removed ATTR{dev_id}=="0x0" and ATTR{type}=="1" from my Ubuntu 14 template.

Some say to remove KERNEL=="eth*" or the entire line is ignored. This was not the case in my setup.

If you 'lose' the MAC address like I did because I rebooted before this step, does not show with ifconfig, go find it in /sys/class/net/assignedName/address.

BTW: this system renamed it eth0, cat /sys/class/net/eth0/address

Step 3:

Assign the new interface name an address

sudo vi /etc/network/interfaces

auto test0
iface test0 inet static
    address 192.168.2.202 --  use your address
    netmask 255.255.255.0 --  use your address
and what other entries your system requires.

Step 4:

reboot (its just easier for most of us)

Now that just gives us a static name for our nic.

You will only add iptable rules to Fedora so this is not need on the Pi.

Assumptions:

Both Fedora and Pi have default routing tables and no iptable rules.

note:

We want to keep our private ip address private and not public.

RFC1918 name    IP address range    largest CIDR block (subnet mask)
24-bit block    10.0.0.0 – 10.255.255.255   10.0.0.0/8 (255.0.0.0)
20-bit block    172.16.0.0 – 172.31.255.255 172.16.0.0/12 (255.240.0.0)
16-bit block    192.168.0.0 – 192.168.255.255   192.168.0.0/16 (255.255.0.0)

Pi:

Assign ip address to usb0

sudo vi /etc/network/interfaces

auto usb0
iface usb0 inet static
    address 172.16.0.1
    netmask 255.240.0.0
add any other values needed.

Fedora:

Enable ipv4 forwarding

sudo vi /etc/sysctl.conf

net.ipv4.ip_forward=1

Assign ip address for test0 (remember we changed the nic name above)

sudo vi /etc/network/interfaces

# This connects to the Pi
auto test0
iface test0 inet static
    address 172.16.0.2
    netmask 255.240.0.0
add any other values needed.

# This is the internet connection
auto wlp4s0
iface wlp4s0 inet static
    address 192.168.2.106
    netmask 255.255.255.255
add any other values needed like
    gateway a.b.c.d
    dns-nameservers 8.8.8.8 8.8.4.4

If wlp4s0 address assigned by DHCP it would look more like this

This is the internet connection

auto wlp4s0
iface wlp4s0 inet dhcp

Set the iptable rules to forward the packets from test0 to wlp4s0 AND wrap the packets with a local subnet addressed... wrapper. Entering rules at the command line.

# this rule will forward all traffic from nic test0 to nic wlp4s0
sudo iptables -A FORWARD -i test0 -o wlp4s0 -j ACCEPT
# this rule will continue to forward any existing connections from test0 to wlp4so
sudo iptables -A FORWARD -i test0 -o wlp4s0 -m state --state ESTABLISHED,RELATED -j ACCEPT

# this rule will wrap the packet with a local address so they do not get lost in transit.
sudo iptables -t nat -A POSTROUTING -j MASQUERADE

note:

No firewall rules are enabled. This is a bare minimum to get it working. Add other rules to secure your system.

Make the iptable rules persistent across reboots.

On Ubuntu16 the package name is iptables-persistent. Fedora may be different.

sudo apt-get install iptables-persistent

Save the current iptable rules

iptables-save > /etc/iptables/rules.v4

Reboot fedora.

Verify:

ip addresses.

iptable rules