1

I have a command which streams data from a server and writes lines to stdout. There is a large number of records that I would like to pipe to less, but I would like to prevent my tool from continuing to stream data until the user goes to the next page, otherwise the tool will continue to download enormous amounts of data while the user is not viewing it yet.

Does less block the input process from writing to stdout until the user goes to the next page, or does less attempt to buffer the entire input somewhere?

If it's the latter, is it possible to change the behavior to the former via less flags, or some implementation details in my code, or do I need to implement my own pagination system?

1 Answer 1

1

Less reads its input lazily. It only reads as much as it needs. If the input is a pipe, the program writing to the pipe will eventually block once it fills the pipe's fixed-size buffer (typically 512B to a few kB).

The laziness stops when the user uses a command that requires getting to the end of the file, such as > or an unsuccessful search. To fulfill such a request, less must read until the end of the file. If the input is infinite (because the program writing to the pipe keeps writing forever), the request will never complete.

If a command takes too long, the user can interrupt it with Ctrl+C. This causes less to close the input, which in turn means that the program writing to the pipe will receive a SIGPIPE signal if the pipe buffer fills up. After interrupting with Ctrl+C, there's no way to resume reading the input if the input comes from a pipe.

6
  • I think the standard OS pipe buffer (for years) is 8k. There might be 8k on both the read side and write side of the pipe, but usually not.
    – user10489
    Dec 7, 2021 at 0:18
  • @user10489 It's classically 512B, still that size on FreeBSD 12. 4kB on modern Linux systems. Dec 7, 2021 at 12:17
  • Odd. BSD 4.2 system I used 25 years ago was 8k. Or maybe it was 4k on each side. At any rate, the buffering should not be less than a page size. Does FreeBSD 12 use 512B pages?
    – user10489
    Dec 7, 2021 at 12:38
  • unix.stackexchange.com/questions/11946/… says it could be up to 64k, although google agrees BSD 12 uses 512b.
    – user10489
    Dec 7, 2021 at 12:46
  • @user10489 There's no “side”: the buffer is attached to the pipe itself, not to each side of the pipe. Are you confusing the in-kernel buffer associated to the pipe with read-ahead buffers and write-pending buffers managed by libc inside each process? The size is not related to MMU pages, if that's the kind of pages you mean. Dec 7, 2021 at 12:49

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .