4

Currently I'm trying to troubleshoot an e-mail server that will allow users to send e-mail, but not log in to the server (POP3) or receive e-mail using their Outlook clients. Previously this was doable, but moving the server to it's current location broke it. For obvious reasons, I suspect it's the firewall causing issues.

Firstly, here is what the handshake (or attempt at) looked like before I made any changes to the firewall:

  • mail-server: 192.168.25.26
  • client: 192.168.25.50

         Source     |     Destination     |   PROTO  |     INFO
    192.168.25.50        192.168.25.26         TCP        50861 > pop3 [SYN] seq=0 win=65535 len=0
    192.168.25.26        192.168.25.50         TCP        pop3 > 58601 [RST, ACK] seq=1 win=1 len=0
    192.168.25.50        192.168.25.26         TCP        [TCP Retransmission] 58061 > pop3 [Syn] etc....
    

and the [rst, ack], [tcp re-transmission] dance happens once more before failing.

The rules for the firewall (Cisco) concerning 192.168.25.26 and POP3 are as follows:

inside:
   source     |     destination     |        service     
     any           192.168.25.0/24          tcp/pop3
192.168.25.0/24          any                tcp/smtp
                  208.105.121.196
outside:
   source     |     destination     |        service     
     any           192.168.25.26            tcp/pop3
                        any                 tcp/smtp

To me, that looks fine. To be certain, on the email server (Zentyal) I ran the following commands:

(IP tables was HEAVILY configured. I suspect the issue lies here. Is there any way to allow all traffic, just for testing's sake?)

sudo iptables -F
sudo iptables -P INPUT ACCEPT
sudo iptables -P FORWARD ACCEPT
sudo iptables -P OUTPUT ACCEPT

However, there's still no outlook connect-ability. At this point, on the email server I ran the command:

telnet 127.0.0.1 110

and was able to log in to the mail server. From a client inside the network, I ran the same command and was told the "connection failed".

After these changes, when I attempt to "test the settings" in Outlook, here's what the handshake looks like. I noticed something peculiar.

     Source     |     Destination     |   PROTO  |     INFO
192.168.25.50        192.168.25.26         TCP        apollo-status > pop3 [SYN] seq=0 win=65535 len=0
192.168.25.26        192.168.25.50         TCP        pop3 > apollo-status [RST, ACK] seq=1 win=1 len=0
192.168.25.50        192.168.25.26         TCP        [TCP Retransmission] apollo-status > pop3 [Syn] etc....

and it does it twice more, like the last time. In the INFO pane, I now see apollo-status instead of a port number. The next two times I ran it, I saw npep_messaging and synapse, respectively. I suspect these are random ports chosen by dovecot, because in dovecot.config I changed the default listener to an asterisk "*" from the port previously specified, which seemed to be an arbitrary port in the 4,000's and had no rules in the firewall.

Just to confirm, it seems the port it uses each time is changing based on the (recommended) configuration. On the loopback interface port 110 clearly has a listener attached to it, but from another computer, it appears to be closed.

Any help at all is appreciated. I've racked my brain with this, and I'm hoping that this community will have some novel ideas.

  • ime RST is usually some sort of network-based filtering. Usually when a host wants to close a connection it would just close the TCP connection or just start dropping packets related to it. – Bratchley Nov 3 '14 at 15:43
  • Hmmm, thanks for letting me know that. I'll have to take another look at the Cisco firewall. However, I can ping the server, ssh into the server, and use the web mail inside the private network. Port 110 is the only trouble port.. – theCowardlyFrench Nov 3 '14 at 23:57
  • Firewalls regularly discriminate based on port numbers. I once had print jobs get an RST but telnet to the MFP would work without issue. It may be some sort of IDS system as well. I'm basically just saying that if it's sending an RST it's most likely because it's not an end-node and can't just close the connection normally. – Bratchley Nov 4 '14 at 14:05
0

so the RST message shouldn't freak you out. It occurs whenever you attempt to access a service that is firewalled or the OS is telling you that there's no such service listening on this port (Incidentally, this trick is used by traceroute to find out when traceroute needs to stop it's probing.).

Let's take a look at what I am TCP sockets are in the Listen state on my box

❯ netstat -an | grep LISTEN                                      [11:49:58 PM]
tcp4       0      0  *.9100                 *.*                    LISTEN
tcp4       0      0  *.17500                *.*                    LISTEN
tcp4       0      0  127.0.0.1.8021         *.*                    LISTEN
tcp6       0      0  ::1.8021               *.*                    LISTEN
tcp4       0      0  127.0.0.1.3306         *.*                    LISTEN
tcp4       0      0  *.22                   *.*                    LISTEN
tcp6       0      0  *.22                   *.*                    LISTEN
tcp4       0      0  127.0.0.1.631          *.*                    LISTEN
tcp6       0      0  ::1.631                *.*                    LISTEN

So let's try to connect to a non existent service and at the same time snoop on my lo traffic.

 ❯ nc -v -z 127.0.0.1 12002                                       [11:50:12 PM]
nc: connectx to 127.0.0.1 port 12002 (tcp) failed: Connection refused

Here's the OS telling me that there's no such service here. Firewalls fake it the exact same way (the nice ones when you use the REJECT destination. Destination DROP just /dev/null your traffic).

❯ sudo tshark -i lo -s0                                                                                                                                [11:50:04 PM]
Capturing on 'Loopback'
  1   0.000000    127.0.0.1 -> 127.0.0.1    TCP 68 54188→12002 [SYN] Seq=0 Win=65535 Len=0 MSS=16344 WS=32 TSval=688032360 TSecr=0 SACK_PERM=1
  2   0.000063    127.0.0.1 -> 127.0.0.1    TCP 44 12002→54188 [RST, ACK] Seq=1 Ack=1 Win=0 Len=0

Connecting to an active port results in the TCP handshake & teardown dance

 ❯ nc -v -z 127.0.0.1 8021                                        [11:50:23 PM]
found 0 associations
found 1 connections:
     1: flags=82<CONNECTED,PREFERRED>
    outif lo0
    src 127.0.0.1 port 54190
    dst 127.0.0.1 port 8021
    rank info not available
    TCP aux info available

And the relevant network traffic:

  3  11.123990    127.0.0.1 -> 127.0.0.1    TCP 68 54190→8021 [SYN] Seq=0 Win=65535 Len=0 MSS=16344 WS=32 TSval=688043453 TSecr=0 SACK_PERM=1
  4  11.124082    127.0.0.1 -> 127.0.0.1    TCP 68 8021→54190 [SYN, ACK] Seq=0 Ack=1 Win=65535 Len=0 MSS=16344 WS=32 TSval=688043453 TSecr=688043453 SACK_PERM=1
  5  11.124091    127.0.0.1 -> 127.0.0.1    TCP 56 54190→8021 [ACK] Seq=1 Ack=1 Win=408288 Len=0 TSval=688043453 TSecr=688043453
  6  11.124099    127.0.0.1 -> 127.0.0.1    TCP 56 [TCP Window Update] 8021→54190 [ACK] Seq=1 Ack=1 Win=408288 Len=0 TSval=688043453 TSecr=688043453
  7  11.136377    127.0.0.1 -> 127.0.0.1    TCP 56 54190→8021 [FIN, ACK] Seq=1 Ack=1 Win=408288 Len=0 TSval=688043464 TSecr=688043453
  8  11.136406    127.0.0.1 -> 127.0.0.1    TCP 56 8021→54190 [ACK] Seq=1 Ack=2 Win=408288 Len=0 TSval=688043464 TSecr=688043464
  9  11.136415    127.0.0.1 -> 127.0.0.1    TCP 56 [TCP Dup ACK 7#1] 54190→8021 [ACK] Seq=2 Ack=1 Win=408288 Len=0 TSval=688043464 TSecr=688043464
 10  11.263097    127.0.0.1 -> 127.0.0.1    TCP 56 8021→54190 [FIN, ACK] Seq=1 Ack=2 Win=408288 Len=0 TSval=688043587 TSecr=688043464
 11  11.263126    127.0.0.1 -> 127.0.0.1    TCP 56 54190→8021 [ACK] Seq=2 Ack=2 Win=408288 Len=0 TSval=688043587 TSecr=688043587

So how do you go about solving your issue?

Ensure that your socket is listening on an externally visible port (*:port_number)

❯ netstat -an | grep LISTEN                                      [11:49:58 PM]
tcp4       0      0  *.9100                 *.*                    LISTEN <<< COOL
tcp4       0      0  *.17500                *.*                    LISTEN
tcp4       0      0  127.0.0.1.8021         *.*                    LISTEN <<< NOT COOL

Start with the simplest firewall or no firewall and connect to the service port from a remote box on the same LAN. Remember to turn on the iptables forwarding kernel flag via sysctl On the server end

sudo iptables  -n -L -X -v
sudo tshark -i eth0 -s0 port 110

on the client end

sudo tshark -i eth0 -s0 port 110
nc  -v -z mailserver.example.org 110

It's important to compare the tshark output from both ends to ensure that what you send is what gets to the remote box

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.