The data that you are feeding through the pipe is not the data of the files that
md5sum is processing, but instead the
md5sum output, which, for every file, consists of one line comprising: the MD5-hash, two spaces, and the file name. Since we know this in advance, can inform
pv accordingly, so as to enable it to display an accurate progress indicator. There are two ways of doing so.
The first, preferred method (suggested by frostschutz) makes use of the fact that
md5sum generates one line per processed file, and the fact that
pv has a line mode that counts lines rather than bytes. In this mode
pv will only move the progress bar when it encounters a newline in the throughput, i.e. per file finished by
md5sum. In Bash, this first method can look like this:
set -- *.iso; md5sum "$@" | pv --line-mode -s $# | sort
set builtin is used to set the positional parameters to the files to be processed (the
*.iso shell pattern is expanded by the shell).
md5sum is then told to process these files (
$@ expands to the positional parameters), and
pv in line mode will move the progress indicator each time a file has been processed / a line is output by
pv is informed of the total number of lines it can expect (
-s $#), as the special shell parameter
$# expands to the number of positional arguments.
The second method is not line-based but byte-based. With
md5sum this unnecessarily complicated, but some other program may not produce lines but for instance continuous data, and then this approach may be more practical. I illustrate it with
md5sum though. The idea is to calculate the amount of data that
md5sum (or some other program) will produce, and use this to inform
pv. In Bash, this could look as follows:
os=$(( $( ls -1 | wc -c ) + $( ls -1 | wc -l ) * 34 ))
md5sum * | pv -s $os | sort
The first line calculates the output size (
os) estimate: the first term is the number of bytes necessary for encoding the filenames (incl. newline), the second term the number of bytes used for encoding the MD5-hashes (32 bytes each), plus 2 spaces. In the second line, we tell
pv that the expected amount of data is
os bytes, so that it can show an accurate progress indicator leading up to 100% (which indicator is updated per finished md5summed file).
Obviously, both methods are only practical in case multiple files are to be processed. Also, it should be noted that since the output of
md5sum is not related to the amount of time the
md5sum program has to spend crunching the underlying data, the progress indicator may be considered somewhat misleading. E.g., in the second method, the file with the shortest name will yield the lowest progress update, even though it may actually be the biggest in size. Then again, if all files have a similar sizes and names, this shouldn't matter much.