I'm trying to limit the number of lines in my log file using:

php $PWD/private_html/run_worker.php | head -n 20000 >> $PWD/private_html/data/logfile.txt 2>&1 &

If I run the command without the "| head -n 20000", my output is being redirected. However, when I include the limiting code, nothing gets written to the log.

Any help would be appreciated.

  • There's probably a better tool for this than head, but what happens when you turn off buffering with e.g. stdbuf ?
    – thrig
    Nov 17, 2016 at 23:00
  • Putting 2>&1 after the head is unlikely to be useful. Did you mean to write a construct formed like php ... 2>&1 | head ... >> logfile?
    – roaima
    Nov 17, 2016 at 23:23
  • @roaima Still no results
    – Eric
    Nov 18, 2016 at 0:35
  • @thrig Sorry for being such a noob, but can you please show me the stdbuf syntax in this case?
    – Eric
    Nov 18, 2016 at 0:36

1 Answer 1


Suffering from Buffering

First off, there may be buffering; by default output to not-terminal-not-standard-error is block buffered (per setvbuf(3)) so one should not expect to see any logs until after a buffer is filled (or not at all when something crashes or eats a fatal signal). This is best tested with two terminals, one to run the commands, and the other to tail -f the output file:

# terminal watcher (how do the log lines appear in the logfile?)
rm logf; touch logf; tail -f logf

# terminal test commands (feed logfile)
php -r 'while(1){echo "logline\n";sleep(1);}' >> logf
perl -E 'while(1){say "logline";sleep 1}' >> logf

From this it appears (at least on my system) that PHP does not block buffer standard out, a departure from the default; however, head does buffer, and stdbuf does not help, as the log lines all appear in a clump:

php -r 'while(1){echo "logline\n";sleep(1);}' | head -7 >> logf
php -r 'while(1){echo "logline\n";sleep(1);}' | stdbuf -o 0 -- head -7 >> logf

So head is probably not the right tool for this task. (As a side-note, various applications will buffer internally, so whatever horrible things stdbuf is doing may not help:

stdbuf -o 0 perl -E 'while(1){say "logline";sleep 1}' >> logf
perl -E 'STDOUT->autoflush;while(1){say "logline";sleep 1}' >> logf

If at all possible, set the appropriate buffering in the application itself.)

As to the Logs

A hard limit on log lines would require something that consumes N lines, but then stays open (or does the application then get killed, or...?):

php -r 'while(1){echo "logline\n";sleep(1);}' 2>&1 \
| perl -ne 'BEGIN{STDOUT->autoflush; $N=7 } if($N>0){print;$N--}' >> logf

Or this could be done internally with PHP, assuming all the logs can be routed through a function to handle this task, or a logging module or framework. Otherwise, there are a bunch of tools that log rotate standard out, with varying levels of options and learning curves involved, though these appear to lack "N lines" and instead set caps based on size.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.