Iptables rule doesn't work

Question

I'm using an application called logstash and I need to receive data from UDP port 514. The problem is that logstash isn't allowed to listen port 514. To solve this problem I decided to use iptables nat table:

iptables -t nat -A PREROUTING -p udp --dport 514 -j REDIRECT --to-port 5140

Afterwards I checked if the packets destined to udp port 514 where being caught in that rule, using the command:

iptables -t nat -nxvL

The output was:

Chain PREROUTING (policy ACCEPT 608395 packets, 392277304 bytes)
    pkts      bytes target     prot opt in     out     source               destination

       1      594 REDIRECT   udp  --  *      *       0.0.0.0/0            0.0.0.0/0            udp dpt:514 redir ports 5140

       0        0 REDIRECT   tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:514 redir ports 5140

Chain INPUT (policy ACCEPT 1716 packets, 638207 bytes)
    pkts      bytes target     prot opt in     out     source               destination

Chain OUTPUT (policy ACCEPT 8906 packets, 538280 bytes)
    pkts      bytes target     prot opt in     out     source               destination

Chain POSTROUTING (policy ACCEPT 8906 packets, 538280 bytes)
    pkts      bytes target     prot opt in     out     source               destination

As you can probably tell the rule only captured one packet, even though there a several reaching that port.

tcpdump -nni any -port 514

the command above has the following output:

12:58:22.222661 IP 100.100.200.2.8587 > 10.93.33.115.514: SYSLOG local7.notice, length: 545
12:58:22.232715 IP 100.100.200.2.10099 > 10.93.33.115.514: SYSLOG local7.notice, length: 519
12:58:22.233787 IP 100.100.200.2.8587 > 10.93.33.115.514: SYSLOG local7.info, length: 699
12:58:22.237041 IP 100.100.200.2.8587 > 10.93.33.115.514: SYSLOG local7.info, length: 550
12:58:22.237100 IP 100.100.200.2.8587 > 10.93.33.115.514: SYSLOG local7.info, length: 564
12:58:22.242670 IP 100.100.200.2.5006 > 10.93.33.115.514: SYSLOG local7.notice, length: 542
12:58:22.242722 IP 100.100.200.2.5006 > 10.93.33.115.514: SYSLOG local7.notice, length: 542
12:58:22.246941 IP 100.100.200.2.8587 > 10.93.33.115.514: SYSLOG local7.warning, length: 746
12:58:22.247627 IP 100.100.200.2.8587 > 10.93.33.115.514: SYSLOG local7.notice, length: 687
12:58:22.247654 IP 100.100.200.2.8587 > 10.93.33.115.514: SYSLOG local7.notice, length: 687
12:58:22.252840 IP 100.100.200.2.8587 > 10.93.33.115.514: SYSLOG local7.notice, length: 604
12:58:22.254676 IP 100.100.200.2.23295 > 10.93.33.115.514: SYSLOG local7.notice, length: 687
12:58:22.254704 IP 100.100.200.2.23295 > 10.93.33.115.514: SYSLOG local7.notice, length: 687
12:58:22.258491 IP 100.100.200.2.8587 > 10.93.33.115.514: SYSLOG local7.notice, length: 677
12:58:22.260588 IP 100.100.200.2.8587 > 10.93.33.115.514: SYSLOG local7.info, length: 581
12:58:22.261878 IP 100.100.200.2.23295 > 10.93.33.115.514: SYSLOG local7.notice, length: 542
12:58:22.261908 IP 100.100.200.2.10099 > 10.93.33.115.514: SYSLOG local7.notice, length: 542
12:58:22.261917 IP 100.100.200.2.23295 > 10.93.33.115.514: SYSLOG local7.notice, length: 540
12:58:22.262554 IP 100.100.200.2.5006 > 10.93.33.115.514: SYSLOG local7.notice, length: 542
12:58:22.262568 IP 100.100.200.2.8587 > 10.93.33.115.514: SYSLOG local7.notice, length: 542
12:58:22.266295 IP 100.100.200.2.5006 > 10.93.33.115.514: SYSLOG local7.notice, length: 639
^C

824 packets captured
855 packets received by filter
18 packets dropped by kernel

Why is the rule not working? Every UDP packet that reaches port 514 should be redirected. I really cant't see the whats causing it. iptables-service is up and running, firewalld is disabled.

Answer 1

This answer from ServerFault applies:

nat table rules always work only for first packet in connection. Subsequent packets of same connection never traverse nat rule list and only supported by conntrack code.

As UDP is connectionless in nature, "connection" here is defined simply by addresses, ports and timeout. So, if second UDP packet with same source port and address and same destination port and address arrives within the timeout, Linux believes it belongs to established "connection" and doensn't evaluate nat rule table for it at all, reusing verdict issued for previous packet.

So even though the rule only captured one packet in your case, all relevant packets were probably redirected. There's nothing to worry about, unless packets don't reach your application (but they should).