Measure pipe throughput in the shell

Question

There is a shell command that allows you to measure how fast the data goes through it, so you can measure the speed of output of commands in a pipe. So instead of:

$ somecommand | anothercommand

you can do something like:

$ somecommand | ??? | anothercommand

And throughput stats (bytes/sec) are printed to stderr, I think. But I can't for the life of me remember what that command was.

Answer 1

cpipe is probably better for these purposes, but another related program is pv (Pipe Viewer):

If you give it the --rate flag it will show the transfer rate

Answer 2

You need a utility called cpipe.

Usage:

tar cCf / - usr | cpipe -vr -vw -vt > /dev/null

Output:

...
  in:  19.541ms at    6.4MB/s (   4.7MB/s avg)    2.0MB
 out:   0.004ms at   30.5GB/s (  27.1GB/s avg)    2.0MB
thru:  19.865ms at    6.3MB/s (   4.6MB/s avg)    2.0MB
... 

Answer 3

As seen at https://askubuntu.com/a/620234, notice that pv, at least, can slow down your throughput significantly. The article linked to covers dd, but the point is that pv can slow down your stuff. If you care, and if you are transferring terabytes of data for example.

Answer 4

If you have Python 2 or 3 and pip (sudo apt-get install python-pip) you can install tqdm:

    python -m pip install tqdm

Then simply:

    somecommand | tqdm | anothercommand

If you need help, run tqdm --help. It has a lot of options. Feel free to read more and make suggestions at https://github.com/tqdm/tqdm

Answer 5

A new tool is now available to inspect already running processes files access, display progress and throughput estimation: https://github.com/Xfennec/cv

If your somecommand or anothercommand are already known by cv, it is just as easy to use as watch cv -wq, or else you have to use -c option to monitor specifically your processes.