3 Clarified packets are discarded NOT rejected
source | link

We have a server (camera) sending RTSP video packets via UDP. At a customer site it travels over several hops, one of which may be an unreliable WiFi link that drops the odd packet or five. Usually this goes unnoticed but sometimes it kills the stream for some seconds and causes customer displeasure (I know, their cr*p network is somehow our problem...)

On testing using tc to simulate a dodgy connection we have found an odd situation: If we break the connection in the return direction (packets silently discarded), after some seconds the flow of UDP packets from our camera stops, even though the RTSP client (Live555 Wis-Streamer) still believes it's merrily squirting UDP packets up the pipe.

This is odd, as obviously UDP packets are not ACK'd and the physical link never drops, so our system has no way of knowing that the packets are dropping into the bit bucket further up the chain and the streamer has no way of knowing that no-one is listening to it (the streamer session timeout does not expire until later).

EDIT: We see ARPing (Who has <client>) at the moment the UDP packets stop coming but none prior to that which would tell the stack the connectivity has dropped.

So I have two questions:

  1. Is there some other mechanism by which the networking stack can tell the connection has issues?
  2. Does the network stack silently drop packets under certain circumstances?

To demonstrate our testing setup:

Normal state, connectivity both ways:

Our server <==> Switch <==> TC  <==> Switch <==> PC
                 |                    |
  Wireshark <-- TAP                   |
                                      |
  Wireshark <----------------------- TAP

Fault state, TC dropping packets going back to our server:

Our server --> Switch --> TC  <==> Switch <==> PC
                 |                    |
  Wireshark <-- TAP                   |
                                      |
  Wireshark <----------------------- TAP

We have a server (camera) sending RTSP video packets via UDP. At a customer site it travels over several hops, one of which may be an unreliable WiFi link that drops the odd packet or five. Usually this goes unnoticed but sometimes it kills the stream for some seconds and causes customer displeasure (I know, their cr*p network is somehow our problem...)

On testing using tc to simulate a dodgy connection we have found an odd situation: If we break the connection in the return direction, after some seconds the flow of UDP packets from our camera stops, even though the RTSP client (Live555 Wis-Streamer) still believes it's merrily squirting UDP packets up the pipe.

This is odd, as obviously UDP packets are not ACK'd and the physical link never drops, so our system has no way of knowing that the packets are dropping into the bit bucket further up the chain and the streamer has no way of knowing that no-one is listening to it (the streamer session timeout does not expire until later).

EDIT: We see ARPing (Who has <client>) at the moment the UDP packets stop coming but none prior to that which would tell the stack the connectivity has dropped.

So I have two questions:

  1. Is there some other mechanism by which the networking stack can tell the connection has issues?
  2. Does the network stack silently drop packets under certain circumstances?

To demonstrate our testing setup:

Normal state, connectivity both ways:

Our server <==> Switch <==> TC  <==> Switch <==> PC
                 |                    |
  Wireshark <-- TAP                   |
                                      |
  Wireshark <----------------------- TAP

Fault state, TC dropping packets going back to our server:

Our server --> Switch --> TC  <==> Switch <==> PC
                 |                    |
  Wireshark <-- TAP                   |
                                      |
  Wireshark <----------------------- TAP

We have a server (camera) sending RTSP video packets via UDP. At a customer site it travels over several hops, one of which may be an unreliable WiFi link that drops the odd packet or five. Usually this goes unnoticed but sometimes it kills the stream for some seconds and causes customer displeasure (I know, their cr*p network is somehow our problem...)

On testing using tc to simulate a dodgy connection we have found an odd situation: If we break the connection in the return direction (packets silently discarded), after some seconds the flow of UDP packets from our camera stops, even though the RTSP client (Live555 Wis-Streamer) still believes it's merrily squirting UDP packets up the pipe.

This is odd, as obviously UDP packets are not ACK'd and the physical link never drops, so our system has no way of knowing that the packets are dropping into the bit bucket further up the chain and the streamer has no way of knowing that no-one is listening to it (the streamer session timeout does not expire until later).

EDIT: We see ARPing (Who has <client>) at the moment the UDP packets stop coming but none prior to that which would tell the stack the connectivity has dropped.

So I have two questions:

  1. Is there some other mechanism by which the networking stack can tell the connection has issues?
  2. Does the network stack silently drop packets under certain circumstances?

To demonstrate our testing setup:

Normal state, connectivity both ways:

Our server <==> Switch <==> TC  <==> Switch <==> PC
                 |                    |
  Wireshark <-- TAP                   |
                                      |
  Wireshark <----------------------- TAP

Fault state, TC dropping packets going back to our server:

Our server --> Switch --> TC  <==> Switch <==> PC
                 |                    |
  Wireshark <-- TAP                   |
                                      |
  Wireshark <----------------------- TAP
2 Clarified a bit, added tag
source | link

We have a server (camera) sending RTSP video packets via UDP. At a customer site it travels over several hops, one of which may be an unreliable WiFi link that drops the odd packet or five. Usually this goes unnoticed but sometimes it kills the stream for some seconds and causes customer displeasure (I know, their cr*p network is somehow our problem...)

On testing using tc to simulate a dodgy connection we have found an odd situation: If we break the connection in the return direction, after some seconds the flow of UDP packets from our camera stops, even though the RTSP client (Live555 Wis-Streamer) still believes it's merrily squirting UDP packets up the pipe.

This is odd, as obviously UDP packets are not ACK'd and the physical link never drops, so our system has no way of knowing that the packets are dropping into the bit bucket further up the chain and the streamer has no way of knowing that no-one is listening to it (the streamer session timeout does not expire until later).

We're not seeing any other trafficEDIT: We see ARPing (ARPing etc.Who has <client>) at the pointmoment the connectivity is lost /UDP packets stop coming but none prior to that which would givetell the game awaystack the connectivity has dropped.

So I have two questions:

  1. Is there some other mechanism by which the networking stack can tell the connection has issues?
  2. Does the network stack silently drop packets under certain circumstances?

To demonstrate our testing setup:

Normal state, connectivity both ways:

Our server <==> Switch <==> TC  <==> Switch <==> PC
                 |                    |
  Wireshark <-- TAP                   |
                                      |
  Wireshark <----------------------- TAP

Fault state, TC dropping packets going back to our server:

Our server --> Switch --> TC  <==> Switch <==> PC
                 |                    |
  Wireshark <-- TAP                   |
                                      |
  Wireshark <----------------------- TAP

We have a server (camera) sending RTSP video packets via UDP. At a customer site it travels over several hops, one of which may be an unreliable WiFi link that drops the odd packet or five. Usually this goes unnoticed but sometimes it kills the stream for some seconds and causes customer displeasure (I know, their cr*p network is somehow our problem...)

On testing using tc to simulate a dodgy connection we have found an odd situation: If we break the connection in the return direction, after some seconds the flow of UDP packets from our camera stops, even though the RTSP client (Live555 Wis-Streamer) still believes it's merrily squirting UDP packets up the pipe.

This is odd, as obviously UDP packets are not ACK'd and the physical link never drops, so our system has no way of knowing that the packets are dropping into the bit bucket further up the chain and the streamer has no way of knowing that no-one is listening to it (the streamer session timeout does not expire until later).

We're not seeing any other traffic (ARPing etc.) at the point the connectivity is lost / packets stop that would give the game away.

So I have two questions:

  1. Is there some other mechanism by which the networking stack can tell the connection has issues?
  2. Does the network stack silently drop packets under certain circumstances?

To demonstrate our testing setup:

Normal state, connectivity both ways:

Our server <==> Switch <==> TC  <==> Switch <==> PC
                 |                    |
  Wireshark <-- TAP                   |
                                      |
  Wireshark <----------------------- TAP

Fault state, TC dropping packets going back to our server:

Our server --> Switch --> TC  <==> Switch <==> PC
                 |                    |
  Wireshark <-- TAP                   |
                                      |
  Wireshark <----------------------- TAP

We have a server (camera) sending RTSP video packets via UDP. At a customer site it travels over several hops, one of which may be an unreliable WiFi link that drops the odd packet or five. Usually this goes unnoticed but sometimes it kills the stream for some seconds and causes customer displeasure (I know, their cr*p network is somehow our problem...)

On testing using tc to simulate a dodgy connection we have found an odd situation: If we break the connection in the return direction, after some seconds the flow of UDP packets from our camera stops, even though the RTSP client (Live555 Wis-Streamer) still believes it's merrily squirting UDP packets up the pipe.

This is odd, as obviously UDP packets are not ACK'd and the physical link never drops, so our system has no way of knowing that the packets are dropping into the bit bucket further up the chain and the streamer has no way of knowing that no-one is listening to it (the streamer session timeout does not expire until later).

EDIT: We see ARPing (Who has <client>) at the moment the UDP packets stop coming but none prior to that which would tell the stack the connectivity has dropped.

So I have two questions:

  1. Is there some other mechanism by which the networking stack can tell the connection has issues?
  2. Does the network stack silently drop packets under certain circumstances?

To demonstrate our testing setup:

Normal state, connectivity both ways:

Our server <==> Switch <==> TC  <==> Switch <==> PC
                 |                    |
  Wireshark <-- TAP                   |
                                      |
  Wireshark <----------------------- TAP

Fault state, TC dropping packets going back to our server:

Our server --> Switch --> TC  <==> Switch <==> PC
                 |                    |
  Wireshark <-- TAP                   |
                                      |
  Wireshark <----------------------- TAP
1
source | link

What is stopping our UDP packets (how does it know the route is down)?

We have a server (camera) sending RTSP video packets via UDP. At a customer site it travels over several hops, one of which may be an unreliable WiFi link that drops the odd packet or five. Usually this goes unnoticed but sometimes it kills the stream for some seconds and causes customer displeasure (I know, their cr*p network is somehow our problem...)

On testing using tc to simulate a dodgy connection we have found an odd situation: If we break the connection in the return direction, after some seconds the flow of UDP packets from our camera stops, even though the RTSP client (Live555 Wis-Streamer) still believes it's merrily squirting UDP packets up the pipe.

This is odd, as obviously UDP packets are not ACK'd and the physical link never drops, so our system has no way of knowing that the packets are dropping into the bit bucket further up the chain and the streamer has no way of knowing that no-one is listening to it (the streamer session timeout does not expire until later).

We're not seeing any other traffic (ARPing etc.) at the point the connectivity is lost / packets stop that would give the game away.

So I have two questions:

  1. Is there some other mechanism by which the networking stack can tell the connection has issues?
  2. Does the network stack silently drop packets under certain circumstances?

To demonstrate our testing setup:

Normal state, connectivity both ways:

Our server <==> Switch <==> TC  <==> Switch <==> PC
                 |                    |
  Wireshark <-- TAP                   |
                                      |
  Wireshark <----------------------- TAP

Fault state, TC dropping packets going back to our server:

Our server --> Switch --> TC  <==> Switch <==> PC
                 |                    |
  Wireshark <-- TAP                   |
                                      |
  Wireshark <----------------------- TAP