0

My iptables keeps growing...

When I do sudo iptables -L | wc -l, it shows a number, if I do it again later that number goes up.

It seems to increase by 1 every 2 seconds.


I tried to reboot, not changing anything...


I'm on Ubuntu 16.04.6

EDIT #1

I did like @Artem mentioned :

sudo iptables-save > /tmp/ipt.dump1
# Wait a few seconds
sudo iptables-save > /tmp/ipt.dump2
diff -u /tmp/ipt.dump1 /tmp/ipt.dump2

And I got this result :

--- /tmp/ipt.dump1      2020-07-20 17:39:27.443308154 +0900
+++ /tmp/ipt.dump2      2020-07-20 17:39:40.831173660 +0900
@@ -1,9 +1,9 @@
-# Generated by iptables-save v1.6.0 on Mon Jul 20 17:39:27 2020
+# Generated by iptables-save v1.6.0 on Mon Jul 20 17:39:40 2020
 *nat
-:PREROUTING ACCEPT [545:78025]
-:INPUT ACCEPT [545:78025]
-:OUTPUT ACCEPT [2686:163879]
-:POSTROUTING ACCEPT [2686:163879]
+:PREROUTING ACCEPT [547:78264]
+:INPUT ACCEPT [547:78264]
+:OUTPUT ACCEPT [2726:166287]
+:POSTROUTING ACCEPT [2726:166287]
 :DOCKER - [0:0]
 -A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER
 -A OUTPUT ! -d 127.0.0.0/8 -m addrtype --dst-type LOCAL -j DOCKER
@@ -12,12 +12,12 @@
 -A POSTROUTING -s 172.17.0.2/32 -d 172.17.0.2/32 -p tcp -m tcp --dport 8501 -j MASQUERADE
 -A DOCKER -i docker0 -j RETURN
 COMMIT
-# Completed on Mon Jul 20 17:39:27 2020
-# Generated by iptables-save v1.6.0 on Mon Jul 20 17:39:27 2020
+# Completed on Mon Jul 20 17:39:40 2020
+# Generated by iptables-save v1.6.0 on Mon Jul 20 17:39:40 2020
 *filter
-:INPUT ACCEPT [11:1274]
+:INPUT ACCEPT [38:5571]
 :FORWARD DROP [0:0]
-:OUTPUT ACCEPT [7:464]
+:OUTPUT ACCEPT [9:720]
 :DOCKER - [0:0]
 :DOCKER-ISOLATION-STAGE-1 - [0:0]
 :DOCKER-ISOLATION-STAGE-2 - [0:0]
@@ -829,6 +829,14 @@
 -A INPUT -s 127.0.0.1/32 -j ACCEPT
 -A INPUT -s 127.0.0.1/32 -j ACCEPT
 -A INPUT -s 127.0.0.1/32 -j ACCEPT
+-A INPUT -s 127.0.0.1/32 -j ACCEPT
+-A INPUT -s 127.0.0.1/32 -j ACCEPT
+-A INPUT -s 127.0.0.1/32 -j ACCEPT
+-A INPUT -s 127.0.0.1/32 -j ACCEPT
+-A INPUT -s 127.0.0.1/32 -j ACCEPT
+-A INPUT -s 127.0.0.1/32 -j ACCEPT
+-A INPUT -s 127.0.0.1/32 -j ACCEPT
+-A INPUT -s 127.0.0.1/32 -j ACCEPT
 -A FORWARD -j DOCKER-USER
 -A FORWARD -j DOCKER-ISOLATION-STAGE-1
 -A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
@@ -841,4 +849,4 @@
 -A DOCKER-ISOLATION-STAGE-2 -j RETURN
 -A DOCKER-USER -j RETURN
 COMMIT
-# Completed on Mon Jul 20 17:39:27 2020
+# Completed on Mon Jul 20 17:39:40 2020

EDIT #2

I followed this to track processes :

git clone https://github.com/brendangregg/perf-tools.git
cd perf-tools
sudo ./execsnoop

I then filtered to see only things related to iptables :

sudo ./execsnoop | grep iptables

And every 1.5 seconds I have 2 new processes :

  8596   8595 iptables -L INPUT
  8599   8534 iptables -I INPUT -s 127.0.0.1 -j ACCEPT
  8705   8704 iptables -L INPUT
  8708   8643 iptables -I INPUT -s 127.0.0.1 -j ACCEPT
  8814   8813 iptables -L INPUT
  8817   8752 iptables -I INPUT -s 127.0.0.1 -j ACCEPT
  8923   8922 iptables -L INPUT
  8926   8861 iptables -I INPUT -s 127.0.0.1 -j ACCEPT
  9033   9032 iptables -L INPUT
  9036   8971 iptables -I INPUT -s 127.0.0.1 -j ACCEPT
  9142   9141 iptables -L INPUT
  9145   9080 iptables -I INPUT -s 127.0.0.1 -j ACCEPT
  9251   9250 iptables -L INPUT
  9254   9189 iptables -I INPUT -s 127.0.0.1 -j ACCEPT
  9360   9359 iptables -L INPUT
  9363   9298 iptables -I INPUT -s 127.0.0.1 -j ACCEPT
  9469   9468 iptables -L INPUT
  9472   9407 iptables -I INPUT -s 127.0.0.1 -j ACCEPT
  9578   9577 iptables -L INPUT
  9581   9516 iptables -I INPUT -s 127.0.0.1 -j ACCEPT
  9687   9686 iptables -L INPUT
  9690   9625 iptables -I INPUT -s 127.0.0.1 -j ACCEPT

But it doesn't help me finding the root cause...

4
  • 1
    Something is adding this rule over and over: -A INPUT -s 127.0.0.1/32 -j ACCEPT This might be a cron job, this might be a buggy user space application (are you running any user space firewalls?), this might be docker, this might be a ufw which has gone bonkers for some reasons. Have you ever tried rebooting? It's unlikely it'll fix the issue but it's worth trying anyways. Also please try stopping docker: sudo systemctl stop docker Commented Jul 20, 2020 at 8:56
  • Thank you for your kind answer. Is there anyway to identify the process doing this ? I tried stopping / uninstalling docker, rebooting, stopping ufw, nothing is working...
    – Astariul
    Commented Jul 20, 2020 at 9:06
  • Please refer to superuser.com/questions/222912/… serverfault.com/questions/199654/… unix.stackexchange.com/questions/260162/… - you need to monitor what launches /sbin/iptables Commented Jul 20, 2020 at 9:22
  • 1
    I've updated the answer to find out what is calling iptables. Please try it. Commented Jul 21, 2020 at 8:00

3 Answers 3

6

Please do the following:

sudo iptables-save > /tmp/ipt.dump1

a few seconds later

sudo iptables-save > /tmp/ipt.dump2

Then please post diff -u /tmp/ipt.dump1 /tmp/ipt.dump2

At least you'll know what you're dealing with.


As a last resort, I'd do the following:

sudo mv /sbin/iptables /sbin/iptables.real

and then create a bash script instead, e.g.

#! /bin/bash
echo "`date`: I was called by $PPID `readlink /proc/$PPID/exe` " >> /tmp/iptables.log
/sbin/iptables.real "$@"

This will let you find out what process is calling iptables incessantly.

1
  • 1
    Thanks for your edit. I used the links you posted as comments, and I could find the PPID of the process that was changing iptables. It didn't really helped me, because the PPID was changing every 2 seconds... Turned out it was a virus... (I added my answer).
    – Astariul
    Commented Jul 22, 2020 at 1:01
2

This awk script might help you find who's calling iptables. It pieces together a process tree from execsnoop's output (pid ppid command ...). The parents are indented and appear after the children, sorry; with more time, I could get it to look like pstree, but this may do.

Run execsnoop for a few seconds with its output diverted to a file, then run this with input from that file.

awk -v search=iptables '
    $1 ~ /^[0-9]+$/ {
        p = $1
        ppid[p] = $2
        # command is in $3..$NF, so we'll
        # remove the first 2 fields, then store $0
        for (i = 3; i <= NF; i++) {
            $(i-2) = $i
        }
        NF -= 2
        cmd[p] = $0
    }
    END {
        for (p in cmd) {
            if (cmd[p] ~ search) {
                prefix = ""
                do {
                    print prefix, p, cmd[p]
                    p = ppid[p]
                    prefix = prefix " "
                } while (p)
            }
        }
    }'
0

Turned out, it was a virus...

I opened another stack exchange post on what to do next.


I'm accepting this answer because it's the answer of my specific case, but be sure to check comments and other answers, which provide really useful tools to track the potential cause of this.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .