5

I can't figure out why the tcp_adv_win_scale and tcp_app_win variables coexist in Linux.  The information from tcp(7) says:

For tcp_adv_win_scale:

tcp_adv_win_scale (integer; default: 2; since Linux 2.4)

    Count buffering overhead as bytes/2^tcp_adv_win_scale, if tcp_adv_win_scale is greater than 0; or bytes-bytes/2^(-tcp_adv_win_scale), if tcp_adv_win_scale is less than or equal to zero.

     The socket receive buffer space is shared between the application and kernel.  TCP maintains part of the buffer as the TCP window, this is the size of the receive window advertised to the other end.  The rest of the space is used as the "application" buffer, used to isolate the network from scheduling and application latencies.  The tcp_adv_win_scale default value of 2 implies that the space used for the application buffer is one fourth that of the total.

And for tcp_app_win:

tcp_app_win (integer; default: 31; since Linux 2.4)

    This variable defines how many bytes of the TCP window are reserved for buffering overhead.

    A maximum of (window/2^tcp_app_win, mss) bytes in the window are reserved for the application buffer.  A value of 0 implies that no amount is reserved.

So I'm not sure of understanding what does tcp_app_win exactly change.  It seems to me that both variables can be used to tweak the TCP application buffer, therefore there is no need of changing them together. I am correct?

2 Answers 2

3

I found this information that talks about tcp_adv_win_scale. The page is titled: TCP performance tuning - how to tune linux.

excerpt

TCP performance is limited by latency and window size (and overhead, which reduces the effective window size) by window_size/RTT (this is how much data that can be "in transit" over the link at any given moment).

To get the actual transfer speeds possible you have to divide the resulting window by the latency (in seconds):

The overhead is: window/2^tcp_adv_win_scale (tcp_adv_win_scale default is 2)

So for linux default parameters for the recieve window (tcp_rmem): 87380 - (87380 / 2^2) = 65536.

Given a transatlantic link (150 ms RTT), the maximum performance ends up at: 65536/0.150 = 436906 bytes/s or about 400 kbyte/s, which is really slow today.

With the increased default size: (873800 - 873800/2^2)/0.150 = 4369000 bytes/s, or about 4Mbytes/s, which is resonable for a modern network. And note that this is the default, if the sender is configured with a larger window size it will happily scale up to 10 times this (8738000*0.75/0.150 = ~40Mbytes/s), pretty good for a modern network.

2.6.17 and later have resonably good defaults values, and actually tune the window size up to the max allowed, if the other side supports it. So since then most of this guide is not needed. For good long-haul throughput the maxiumum value might need to be increased though.

I was able to follow that but didn't quite understand the relationship, if any, between these 2 variables.

I only marginally understood what that was trying to explain. At the core it sounds like this parameter to scale the amount of buffering space is to be used for TCP and for the application.

Searching a bit more I found these explanations which made more sense. The page was titled: Ipsysctl tutorial 1.0.4 - Chapter 3. IPv4 variable reference.

excerpt

3.3.2. tcp_adv_win_scale

This variable is used to tell the kernel how much of the socket buffer space should be used for TCP window size, and how much to save for an application buffer. If tcp_adv_win_scale is negative, the following equation is used to calculate the buffer overhead for window scaling:

Where bytes are the amount of bytes in the window. If the tcp_adv_win_scale value is positive, the following equation is used to calculate the buffer overhead:

The tcp_adv_win_scale variable takes an integer value and is per default set to 2. This in turn means that the application buffer is 1/4th of the total buffer space specified in the tcp_rmem variable.

3.3.3. tcp_app_win

This variable tells the kernel how many bytes to reserve for a specific TCP window in the TCP sockets memory buffer where the specific TCP window is transfered in. This value is used in a calculation that specifies how much of the buffer space to reserve that looks as the following:

ss of formula

As you may understand from the above calculation, the larger this value gets, the smaller will the buffer space be for the specific window. The only exception to this calculation is 0, which tells the kernel to reserve no space for this specific connection. The default value for this variable is 31 and should in general be a good value. Do not change this value unless you know what you are doing.

Based on these explanations it sounds like the first parameter, tcp_adv_win_scale is controlling the split of the socket buffer space in terms of how it get's carved up for TCP window use vs. application buffer.

Whereas the 2nd parameter, tcp_app_win is specifying the number of bytes to reserve for the application buffer mentioned in the tcp_adv_win_scale description.

0

Here are my findings from kernel sources:

I would say that logic is not to reserve buffer for application in a new connections and grow it upon receiving data.
That way initial rwnd theoretically could be advertised as big as the whole buffer is. Sender will send that much of data. Receiver buffer will fill up, receiver will shrink win and increase application portion of buffer according to tcp_adv_win_scale.
So, as mentioned in @slm answer - there seems to be no reason for touching tcp_app_win.

1
  • It would be helpful if you gave a link to the source code you are talking about. … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … Please do not respond in comments; edit your answer to make it clearer and more complete. Commented Jan 27, 2022 at 15:01

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .