I would like to track progress of a slow operation using
pv. The size of the input of this operation is known in advance, but the size of its output is not. This forced me to put
pv to the left of the operation in the pipe.
The problem is that the long-running command immediately consumes its whole input because of buffering. This is somewhat similar to the Turn off buffering in pipe question, but in my case it is the consuming operation that is slow, not the producing one and none of the answers to the other question seem to work in this case.
Here is a simple example demonstrating the problem:
seq 20 | pv -l -s 20 | while read line; do sleep 1; done 20 0:00:00 [13.8k/s] [=====================================>] 100%
Instead of getting updated every second, the progress bar immediately jumps to 100% and stays there for the entire 20 seconds it takes to process the input.
pv could only measure the progress if the lines were processed one by one, but the entire input of the last command seems to be read into a buffer.
A somewhat longer example that also demonstrates the unknown number of output lines:
#! /bin/bash limit=10 seq 20 | \ pv -l -s 20 | \ while read num do sleep 1 if [ $num -gt $limit ] then echo $num fi done
Any suggestions for a workaround? Thanks!