I am looking at how HTTP proxies and reverse proxies deal with slow client problems. The idea is that the upstream server only have a limited slots for clients and if the client is slow to receive data, it consumes a slot for a long time. A reverse proxy can be used to buffer the response, free the slot earlier on upstream and then forward slowly the response to the client.
For instance, nginx suggests to enable upstream response buffering by allocating (by default) up to 8 buffers of 8k each. If those buffers are filled, it can start buffering on the disk (but I disabled this feature, my disks are busy enough).
However, I did multiple checks and it seems that the kernel allocates a quite large RCVBUF (receive buffer) of around 1-4MB. If upstream sends a response of 2MB while the end client don't read anything, the proxy buffers will be filled soon, and the kernel buffer will be used instead.
Since the proxy will buffer less data than the kernel, I don't see how it helps to deal with slow clients. What can be the advantages of explicitly implementing/enabling a buffering feature in the proxy while the kernel does enough for us?
Edit: after the first response, I would like to give some details about what I tested.
- a client program connects to the reverse proxy, waits during a few seconds and starts reading.
- the reverse proxy only buffers up to 8kB in user space memory, after a read(), it will log the size of the socket's receive buffer, .
- upstream serves a HTTP response of 2MB (plus headers), the log the time it took between accept() and close().
When testing, I can see that the server will never wait on a write(), and even call close() before the slow client performed the first read(). Also, the size of the socket receive buffer will grow and exceed 2MB: the whole response from the server will be buffered.
I ran the tests with the upstream server on the same host than the client and proxy, and with the upstream on a distant host, the observed behavior is the same.
Also, I understand that the kernel may use smaller buffers under memory pressure, but this affects the reverse proxy too (which may thus be unable to buffer the response in user space).