41

There is a shell command that allows you to measure how fast the data goes through it, so you can measure the speed of output of commands in a pipe. So instead of:

$ somecommand | anothercommand

you can do something like:

$ somecommand | ??? | anothercommand

And throughput stats (bytes/sec) are printed to stderr, I think. But I can't for the life of me remember what that command was.

6 Answers 6

60

cpipe is probably better for these purposes, but another related program is pv (Pipe Viewer):

Screenshot of pv from the pv homepage

If you give it the --rate flag it will show the transfer rate

0
20

You need a utility called cpipe.

Usage:

tar cCf / - usr | cpipe -vr -vw -vt > /dev/null

Output:

...
  in:  19.541ms at    6.4MB/s (   4.7MB/s avg)    2.0MB
 out:   0.004ms at   30.5GB/s (  27.1GB/s avg)    2.0MB
thru:  19.865ms at    6.3MB/s (   4.6MB/s avg)    2.0MB
... 
2
  • 3
    No longer found any valid reference to cpipe... but pv is equivalent. Commented Jan 7, 2015 at 8:26
  • 1
    Is there a similar command for windows, or is there any pv that is windows compatible ?
    – Radon8472
    Commented Oct 27, 2020 at 22:36
3

As seen at https://askubuntu.com/a/620234, notice that pv, at least, can slow down your throughput significantly. The article linked to covers dd, but the point is that pv can slow down your stuff. If you care, and if you are transferring terabytes of data for example.

2

If you have Python 2 or 3 and pip (sudo apt-get install python-pip) you can install tqdm:

    python -m pip install tqdm

Then simply:

    somecommand | tqdm | anothercommand

If you need help, run tqdm --help. It has a lot of options. Feel free to read more and make suggestions at https://github.com/tqdm/tqdm

0

A new tool is now available to inspect already running processes files access, display progress and throughput estimation: https://github.com/Xfennec/cv

If your somecommand or anothercommand are already known by cv, it is just as easy to use as watch cv -wq, or else you have to use -c option to monitor specifically your processes.

0

Depending on the scenario, I can think of two different approaches:

  1. For a progress bar showing the progress/throughput of something, use pv.
  • E.x. pv my-huge-nix-iso.iso > /dev/sdh
  • E.x. echo | b3sum -l $((1024*1024*1024*10)) --raw | pv >/dev/null
  • Install via apt install pv
  • For a fixed-size, add -Ss <bytes>. E.x. pv -Ss $((1024*1024*1024)) /dev/urandom >/dev/null
  • Make sure to add --no-splice if outputting to a block device such as a USB drive or harddisk, otherwise pv will load gigabytes of input file into a kernel dirty writeback buffer.
  1. For benchmarking the stdout throughput of a program generating endless output, I recommend the following one-liner BASH. It divides the number of generated bytes in two seconds by the user time measured by time (that excludes system time spent in syscalls) and uses kill signal 9 to instantly destroy the process so no user time is wasted on cleanup. Its precision seems to be +/-10%.
dc -e "4 k $(export TIMEFORMAT=%U; { time timeout -s9 --foreground 2 <COMMAND> | wc -c; } 2>&1 | tail -n2) / p" | numfmt --to=iec --format="%.3fiB/s"

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .