1

I have home a server that used to have one NIC, but I added a second one to give a dedicated link to the TV tuner.

I went from having eth0 (the onboard NIC) to enp7s0 (the new NIC) and enp9s1 (the new name for the onboard NIC).

All the services were updated to use the new interface names.

My iptables script was updated to use the new interface names on adapter specific rules (everything uses enp9s1, except the DHCP server and a UDP port on enp7s0)

My router configuration didn't change, and since the MAC address hasn't changed, it should still give the NIC the same IP address (and it does).

But, I'm baffled as to why everything works locally and suddenly nothing works over NAT. I can see the router is forwarding packets, but the server still rejects them. Even after temporarily disabling the firewall entirely, it still doesn't work. All I did was add a NIC.

So, how do I get the server to accept NATed packets again?

Attached is my firewall script:

#!/bin/bash

# ============================
#    Command Aliases
#    (for faster typing)
# ============================
ip4='/sbin/iptables'
ip6='/sbin/ip6tables'

# logical interface names
nicmobo='enp9s1'
nicext1='enp7s0'

# ============================
#    Flush rules, delete
#    chains, zero counters
# ============================
$ip4 -F;$ip4 -X;$ip4 -Z
$ip6 -F;$ip6 -X;$ip6 -Z

# ============================
#    Default Rules
# ============================

# ----------------------
#   IPv4
# ----------------------

# Drop all incoming connections by default
$ip4 -P INPUT DROP
$ip4 -P FORWARD DROP
$ip4 -P OUTPUT ACCEPT

# Allow all connections from loopback intrface
$ip4 -A INPUT -i lo -j ACCEPT
# Allow all related/established incoming connections
$ip4 -A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT

# ----------------------
#   IPv6
# ----------------------
# Drop all incoming by default, unrestrict output
$ip6 -P INPUT DROP
$ip6 -P FORWARD DROP
$ip6 -P OUTPUT ACCEPT

# Allow loopback interface
$ip6 -A INPUT -i lo -j ACCEPT

# Reject connection attempt not initiated from the host
# $ip6 -A INPUT -p tcp --syn -j DROP

# Allow return connections initiated from the host
$ip6 -A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT

# Acceot ICMPv6 packets
$ip6 -A INPUT -p ipv6-icmp -j ACCEPT


# ============================
#   SSH Connections on mobo
# ============================

# Allow incoming SSH connections on the ssh listening port (currently
# port 22 at the time of writing this)
$ip4 -A INPUT -p tcp -m tcp --dport 22 -j ACCEPT
$ip6 -A INPUT -p tcp -m tcp --dport 22 -j ACCEPT
# Also allow SSH on 2322 from the local LAN
$ip4 -A INPUT -s 172.26.62.0/23 -p tcp -m tcp -m state --state NEW --dport 2322 -j ACCEPT


# ============================
#   VPN Ports on mobo
# ============================

# Allow L2TP/IPSEC UDP Ports 500, 4500
$ip4 -i $nicmobo -A INPUT -m state --state NEW -m udp -p udp --dport 500 -j ACCEPT
$ip6 -i $nicmobo -A INPUT -m state --state NEW -m udp -p udp --dport 500 -j ACCEPT
$ip4 -i $nicmobo -A INPUT -m state --state NEW -m udp -p udp --dport 4500 -j ACCEPT
$ip6 -i $nicmobo -A INPUT -m state --state NEW -m udp -p udp --dport 4500 -j ACCEPT
# Allow inbound TCP 992 for VPN connections
$ip4 -i $nicmobo -A INPUT -m state --state NEW -m tcp -p tcp --dport 992 -j ACCEPT
$ip6 -i $nicmobo -A INPUT -m state --state NEW -m tcp -p tcp --dport 992 -j ACCEPT
# Allow inbound TCP 1194 for TCP openVPN connections
$ip4 -i $nicmobo -A INPUT -m state --state NEW -m tcp -p tcp --dport 1194 -j ACCEPT
$ip6 -i $nicmobo -A INPUT -m state --state NEW -m tcp -p tcp --dport 1194 -j ACCEPT
# Allow inbound UDP 1194 for openVPN connections
$ip4 -i $nicmobo -A INPUT -m state --state NEW -m udp -p udp --dport 1194 -j ACCEPT
$ip6 -i $nicmobo -A INPUT -m state --state NEW -m udp -p udp --dport 1194 -j ACCEPT
# Allow inbound TCP 5555 for TCP VPN connections
$ip4 -i $nicmobo -A INPUT -m state --state NEW -m tcp -p tcp --dport 5555 -j ACCEPT
$ip6 -i $nicmobo -A INPUT -m state --state NEW -m tcp -p tcp --dport 5555 -j ACCEPT


# ============================
#   Emby HTTPS Non-standard Port (8920) on mobo
# ============================
$ip4 -i $nicmobo -A INPUT -m state --state NEW -m tcp -p tcp --dport 8920 -j ACCEPT
$ip6 -i $nicmobo -A INPUT -m state --state NEW -m tcp -p tcp --dport 8920 -j ACCEPT


# ===========================
#  HDHomerun Discovery Ports Allow from prvate lan only (second NIC)
# ===========================
$ip4 -i $nicext1 -A INPUT -s 172.27.0.0/28 -p udp -m state --state NEW -m udp --sport 65001 -j ACCEPT


# ===========================
#  Port 80, 443 on mobo
# ===========================
$ip4 -i $nicmobo -A INPUT -m state --state NEW -m tcp -p tcp --dport 80 -j ACCEPT
$ip6 -i $nicmobo -A INPUT -m state --state NEW -m tcp -p tcp --dport 80 -j ACCEPT
$ip4 -i $nicmobo -A INPUT -m state --state NEW -m tcp -p tcp --dport 443 -j ACCEPT
$ip6 -i $nicmobo -A INPUT -m state --state NEW -m tcp -p tcp --dport 443 -j ACCEPT

# ===========================
#  SAMBA on Local NETWORK on mobo
# ===========================
$ip4 -i $nicmobo -A INPUT -s 172.26.62.0/23 -p tcp -m state --state NEW -m tcp --dport 139 -j ACCEPT
$ip4 -i $nicmobo -A INPUT -s 172.26.62.0/23 -p tcp -m state --state NEW -m tcp --dport 445 -j ACCEPT
$ip4 -i $nicmobo -A INPUT -s 172.26.62.0/23 -p udp -m state --state NEW -m udp --dport 137 -j ACCEPT
$ip4 -i $nicmobo -A INPUT -s 172.26.62.0/23 -p udp -m state --state NEW -m udp --dport 138 -j ACCEPT

# ===========================
#  Allow DHCP to private LAN (second NIC)
# ===========================
$ip4 -i $nicext1 -A INPUT -s 172.27.0.0/28 -p udp -m state --state NEW -m udp --dport 67 --sport 68 -j ACCEPT


exit 0

Per request, here's my ip -4 route list output:

default via 172.27.0.14 dev enp7s0 proto static 
default via 172.26.62.1 dev enp9s1 proto dhcp src 172.26.62.30 metric 100 
172.26.62.0/23 dev enp9s1 proto kernel scope link src 172.26.62.30 
172.26.62.1 dev enp9s1 proto dhcp scope link src 172.26.62.30 metric 100 
172.27.0.0/28 dev enp7s0 proto kernel scope link src 172.27.0.14 

Also, here's my iptables-save -c output:

# Generated by iptables-save v1.6.1 on Wed May 22 11:26:32 2019
*filter
:INPUT DROP [28356:9869864]
:FORWARD DROP [0:0]
:OUTPUT ACCEPT [2899:379735]
[894:99268] -A INPUT -i lo -j ACCEPT
[1830:147589] -A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
[0:0] -A INPUT -p tcp -m tcp --dport 22 -j ACCEPT
[0:0] -A INPUT -s 172.26.62.0/23 -p tcp -m tcp --dport 2322 -m state --state NEW -j ACCEPT
[0:0] -A INPUT -i enp9s1 -p udp -m state --state NEW -m udp --dport 500 -j ACCEPT
[0:0] -A INPUT -i enp9s1 -p udp -m state --state NEW -m udp --dport 4500 -j ACCEPT
[0:0] -A INPUT -i enp9s1 -p tcp -m state --state NEW -m tcp --dport 992 -j ACCEPT
[0:0] -A INPUT -i enp9s1 -p tcp -m state --state NEW -m tcp --dport 1194 -j ACCEPT
[0:0] -A INPUT -i enp9s1 -p udp -m state --state NEW -m udp --dport 1194 -j ACCEPT
[0:0] -A INPUT -i enp9s1 -p tcp -m state --state NEW -m tcp --dport 5555 -j ACCEPT
[4:240] -A INPUT -i enp9s1 -p tcp -m state --state NEW -m tcp --dport 8920 -j ACCEPT
[0:0] -A INPUT -s 172.27.0.0/28 -i enp7s0 -p udp -m state --state NEW -m udp --sport 65001 -j ACCEPT
[0:0] -A INPUT -i enp9s1 -p tcp -m state --state NEW -m tcp --dport 80 -j ACCEPT
[0:0] -A INPUT -i enp9s1 -p tcp -m state --state NEW -m tcp --dport 443 -j ACCEPT
[2:128] -A INPUT -s 172.26.62.0/23 -i enp9s1 -p tcp -m state --state NEW -m tcp --dport 139 -j ACCEPT
[4:256] -A INPUT -s 172.26.62.0/23 -i enp9s1 -p tcp -m state --state NEW -m tcp --dport 445 -j ACCEPT
[36:3312] -A INPUT -s 172.26.62.0/23 -i enp9s1 -p udp -m state --state NEW -m udp --dport 137 -j ACCEPT
[20:4724] -A INPUT -s 172.26.62.0/23 -i enp9s1 -p udp -m state --state NEW -m udp --dport 138 -j ACCEPT
[9:2952] -A INPUT -s 172.27.0.0/28 -i enp7s0 -p udp -m state --state NEW -m udp --sport 68 --dport 67 -j ACCEPT
COMMIT
# Completed on Wed May 22 11:26:32 2019

I think my default routes are the issue. But, how to fix, and make persistent?

--- EDIT 5/22/19 07:40am CDT --- One NIC, enp7s0, is an adapter used to create a new /28 LAN with no Internet access (no NAT needed). It hosts the 172.27.0.0/28 network. It also runs a DHCP server, since the devices I'm using don't support static IPs (I set them via address reservations in the DHCP config instead). The other NIC, enp9s1, is the one with the DHCP client, internet access, and all the "regular" stuff. I think I need to just clear the default route for the other NIC altogether.

I deleted the default route for enp7s0, and now I can access the server remotely, and I verified everything else is still working as necessary. Now to reboot and check if it re-adds itself, and if so, make it not.

migrated from serverfault.com May 22 at 16:55

This question came from our site for system and network administrators.

  • indeed: the first default route is applied: through the new NIC. If both NICs require some unknown internet access then it's the multi-homed problem usually solved with policy routing. If you know that only a few remote IPs are needed through the new NIC, then don't use a default route there. Anyway you can't have two default routes in the same table and both used (policy routing uses multiple tables, but still, only one default route per table) – A.B May 22 at 17:37
  • One NIC, enp7s0, is an adapter used to create a new /28 LAN with no Internet access (no NAT needed). It hosts the 172.27.0.0/28 network. It also runs a DHCP server, since the devices I'm using don't support static IPs (I set them via address reservations in the DHCP config instead). The other NIC, enp9s1, is the one with the DHCP client, internet access, and all the "regular" stuff. I think I need to just clear the default route for the other NIC altogether. – Adam Harbach May 23 at 12:25
2

Wow, I know a lot of Linux, and there's still a lot more to learn…

I had to edit the netplan config file (the program that replaces ifupdown in Ubuntu) and remove the gateway from my new NIC. I thought it needed one just out of habbit, but since it has no Internet access, it doesn't need a gateway.

That meant in the shell…

nano /etc/netplan/01-netcfg.yaml

And remove the very last line:

# This file describes the network interfaces available on your system
# For more information, see netplan(5).

#enp9s1 - onboard adapter
#enp7s0 - addon nic


# old config
# (Commented out old config)
# end old config

# begin new config

network:
  version: 2
  renderer: networkd
  ethernets:
    enp9s1:
      dhcp4: yes
    enp7s0:
      dhcp4: no
      addresses: [172.27.0.14/28]
      gateway4: 172.27.0.14

That keeps the unnecessary static route from being created.

Then I ran two more commands, the first to create all the config files (I haven't researched what all happens under the hood of netplan, but it was necessary). The second commits the changes to the system without having to reboot, changing all the network settings to match your config file. In this case, it deleted the default gateway.

netplan generate
netplan apply

Subsequent reboots will use the config file and the routes will stay the same.

Now, every reboot, my routes are:

default via 172.26.62.1 dev enp9s1 proto dhcp src 172.26.62.30 metric 100 
172.26.62.0/23 dev enp9s1 proto kernel scope link src 172.26.62.30 
172.26.62.1 dev enp9s1 proto dhcp scope link src 172.26.62.30 metric 100 
172.27.0.0/28 dev enp7s0 proto kernel scope link src 172.27.0.14 

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.