5

I started to add some basic iptables rules on my Debian Jessie server. My objective is to filter and log network traffic (for security and learning purposes). Disregarding ICMP packets, these are the rules I'm using:

# INPUT
-A INPUT -i lo -j ACCEPT
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A INPUT -p tcp -m tcp --dport 22 -j ACCEPT
-A INPUT -p tcp -j REJECT --reject-with tcp-reset

# OUTPUT
-A OUTPUT -o lo -j ACCEPT
-A OUTPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A OUTPUT -p udp -m udp --dport 53 -j ACCEPT
-A OUTPUT -p tcp -m tcp --dport 25 -j ACCEPT
-A OUTPUT -m limit -j LOG --log-prefix "UNKNOWN_OUTGOING: " --log-level 5

The policy is set to ACCEPT for both INPUT and OUTPUT.

Now the log frequently lists outgoing RST packets, usually to port 80. The SRC IP here belongs to my server, the destination IP is partially edited out to not disclose other people activities.

Aug 14 11:48:37 reynholm kernel: [81795.100496] UNKNOWN_OUTGOING: IN= OUT=ifext SRC=89.238.65.123 DST=108.162.[edited] LEN=40 TOS=0x00 PREC=0x00 TTL=64 ID=0 DF PROTO=TCP SPT=3594 DPT=80 WINDOW=0 RES=0x00 RST URGP=0

I don't understand what's causing this, there are no applications running other than SSH and an MTA. Is it because of my input REJECT rule? But shouldn't those packets then be handled by the output state rule?

Below is a capture of one of those packets together with the connection attempt apparently triggering it. No packet was sent between my server and 108.162.[edited] before this.

11:48:37.860337 IP (tos 0x0, ttl 60, id 0, offset 0, flags [DF], proto TCP (6), length 44)
    108.162.[edited].80 > 89.238.65.123.3594: Flags [S.], cksum 0x79bb (correct), seq 79911989, ack 235561828, win 29200, options [mss 1460], length 0
        0x0000:  4500 002c 0000 4000 3c06 7342 6ca2 0000  E..,..@.<.sBl...
        0x0010:  59ee 417b 0050 0e0a 04c3 5c35 0e0a 6364  Y.A{.P....\5..cd
        0x0020:  6012 7210 79bb 0000 0204 05b4 0000       `.r.y.........
11:48:37.860408 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 40)
    89.238.65.123.3594 > 108.162.[edited].80: Flags [R], cksum 0x648e (correct), seq 235561828, win 0, length 0
        0x0000:  4500 0028 0000 4000 4006 6f46 59ee 417b  E..(..@.@.oFY.A{
        0x0010:  6ca2 0000 0e0a 0050 0e0a 6364 0000 0000  l......P..cd....
        0x0020:  5004 0000 648e 0000                      P...d...
  • Is it possible that your system is trying to connect to Debian repositories? – jcbermu Aug 14 '15 at 7:58
  • @jcbermu It does, but I specifically checked that none of the packages in question is to a Debian repository server IP. – tarleb Aug 14 '15 at 8:00
  • 1
    have you captured the packets and inspected them – gwillie Aug 14 '15 at 8:02
  • @gwillie Added a capture. – tarleb Aug 14 '15 at 12:03
  • @roaima Both have ACCEPT policies – tarleb Aug 14 '15 at 13:05
6

The creation of the TCP RST packet is from your rule

-A INPUT -p tcp -j REJECT --reject-with tcp-reset

The default policy (ACCEPT in your case) only applies to packets that do not match any of the rules in your chain. If a packet matches the rule above with the REJECT target, it will not be subject to the default policy and will be REJECTed (and generate a TCP RST) rather than ACCEPTed.

This TCP RST will not match your rule:

-A OUTPUT -m state --state RELATED,ESTABLISHED -j ACCEPT

because it is not RELATED to another established connection and it is not part of an ESTABLISHED connection. It will continue through your rules and match

-A OUTPUT -m limit -j LOG --log-prefix "UNKNOWN_OUTGOING: " --log-level 5

and end up in your log. If you do not want to log these RST packets, either adjust this rule to not match them or insert an earlier rule to match the RST packets and so something with them before they get here.


Something else I'm noticing is that the first packet you are logging is a SYN/ACK packet from a remote webserver, which looks like a response packet from the remote webserver to a SYN packet you would have earlier sent to begin the connection to the remote host on port 80. If you didn't send an initial SYN, I don't think the connection would match 'ESTABLISHED', but if you did send a SYN then I think the connection should mach 'ESTABLISHED'. This could be messing with which rule your RST ends up matching.

  • I'm not sure that this is all there is to it. Most connections are reset without a message in my log: grep -c UNKNOWN /var/www/kern.log gives me 69, while iptables -vL says that 2691 packages matched the REJECT rule. According to the above, shouldn't those numbers be equal? – tarleb Aug 14 '15 at 14:14
  • @tarleb by default the limit matcher will only match "3 packets/hour with a burst of 5". You are only actually logging a small subset of the RST packets, which explains your observation. – casey Aug 14 '15 at 14:18
  • From iptables -nvL: Chain OUTPUT (policy ACCEPT 11 packets, 460 bytes), so I don't think that's the cause for the discrepancy. I'll crank up the logging (--limit 9/min) to make sure. – tarleb Aug 14 '15 at 14:33
  • 1
    @tarleb from your packet log above, is there an earlier packet from your host to that remote host on port 80? The first packet you show from them:80 -> you:3594 has both SYN and ACK flags set, which make it look like a reply to a SYN packet you would have first sent. I'm just trying to understand what is going on but on the surface it looks like your machine tried to initiate an outbound connection and when the remote replied to you, you respond with a RST closing the connection instead of ACK to establish it. – casey Aug 14 '15 at 15:30
  • 1
    @tarleb i made an edit. I'm not 100% on how state will match things as all my rules use conntrack and I have explicit rules for NEW connections. – casey Aug 14 '15 at 17:00
3

Extending the great answer by @casey: The cause for this is that the package from remote has the SYN and ACK flags set. This is INVALID for a first package. iptables does not consider the reset packet as RELATED to the initial packet, as it clearly makes no sense.

The effect can be reproduced using a tool like hping3: A SYN/ACK packet is sent with hping3 -c 1 --syn --ack 89.238.65.123. The server's response to this packet will not have the ACK flag set and will not match the state rule (i.e. is not related to the send ping). A log entry is triggered as a result.

No such effect occurs if the ACK flag is unset in the initial request.

I got rid of the log messages by filtering and dropping incoming INVALID tcp packets.

-I INPUT 3 -m state --state INVALID -j DROP
2

This looks like it could be part of a port scan initiated from tcp/80 on the remote host. The remote host probes your service tcp/3594 and your system correctly says "No. Now go away (RST)". Unfortunately fairly normal stuff for Internet connecting systems.

You might like to install fail2ban, which can be very good at blocking hosts trying to connect to your system in an unacceptable manner. A warning, though: configuring it beyond the included examples is not trivial.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.