0

I am using PV -n command to read partitions and using gzip with pipe to compress the read data and storing the file. While data is read and written I am using a while loop to show progress using linux dialog utility.

This works great, progress is updated. I want to also display the transfer speed / read speed in Mbits. Also I want to update the DB table with progress as well as transfer speed in Mbits.

Since I am using a while loop to read each line, progress should be displayed on a new line for every update. See below my code.

        (pv -n /dev/$partitions | gzip -c >$path/${filename// /_}-${partitions}.img.gz) 2>&1 | while IFS= read -r progress;
                                do
                                        echo "processing /dev/$partitions currently completed $progress" >/run/log.log
                                        echo $progress | dialog --title "Capturing OS image of $hdd" --gauge " now creating image of HDD $hdd writing $filename Image, please wait...\n\n\n
                                        Processing Partition $i of $totalparts\n\n
                                        This process may take some time to complete\n\n" 13 90 0

mysql -u root -pxxxxxx dbi -h localhost | insert into speed(progress, speed) Values ("$line", "mbits")
                                    done

if I use pv -n command it only returns numeric value of progress on a newline. See below example:

( /data/pv -n /dev/nvme0n1p1 | gzip -c >/run/test.img ) 
5
9
29
67
100

Above works great for the progress bar, but I want to do update my db with average speed in mbits.

When I run pv command for progress with average speed agrument, progress is updated on the same line instead of print new lines and it breaks my script. See below example.

(pv -rep /dev/nvme0n1p1 | gzip -c >/run/test.img ) 
[4.9MiB/s] [====>                             ]  4% ETA 0:00:19

ideal output should be like this.

(pv -rep /dev/nvme0n1p1 | gzip -c >/run/test.img ) 
[ 4.18MiB/s] [====>                          ]  14% ETA 0:00:19
[14.49MiB/s] [===========>                   ]  54% ETA 0:00:19
[24.39MiB/s] [========================>      ]  74% ETA 0:00:19
[44.29MiB/s] [===========================>   ]  78% ETA 0:00:19
[46.19MiB/s] [=============================> ]  98% ETA 0:00:19
[57.99MiB/s] [==============================>]  100% ETA 0:00:19

I can use AWK, Sed and Grep to format the the required data and use it in my while loop. But how can I get this to work.

If I use pv -F $'%t %r %e\n' I get the desired results. But I cannot use AWK, grep or tr commands. See below example, it returns nothing.

(pv -F $'%t %r %e\n' /dev/nvme0n1p1 | gzip -c >/run/test.img ) 2>&1 | tr -d ':[]'

Also if I don't redirect stderr to stdout, with the same command above, I don't get desired results, see below, using tr -d to delete following characters ":[]" but does not work.

 (pv -F $'%t %r %e\n' /dev/nvme0n1p1 | gzip -c >/run/test.img ) | tr -d ':[]'
0:00:01 [25.2MiB/s] ETA 0:00:18
0:00:02 [23.7MiB/s] ETA 0:00:18
0:00:03 [ 100MiB/s] ETA 0:00:07
0:00:04 [ 199MiB/s] ETA 0:00:01

If I use other arguments such as pv -n -r -e, it ignores all other parameters just returns the numeric progress value on a newline.

Maybe there is an alternative to pv that I can use to achieve exactly described above or maybe someone can help with pv command.

2
  • Please clarify your specific problem or provide additional details to highlight exactly what you need. As it's currently written, it's hard to tell exactly what you're asking.
    – Community Bot
    Sep 22, 2021 at 12:13
  • I have updated the question with more clarity Sep 22, 2021 at 21:28

1 Answer 1

0

Use -F to format the output:

For example:

pv -F "%T %t %r %e"

Will give you Percentage time-elapsed rate and ETA

3
  • The problem with pv -F "%T %t %r %e" is that it update the data on the same line, I want the progress on newline, so that I can continue to execute my while loop. Sep 22, 2021 at 12:24
  • @user2107349, give it a hard newline in the format string with pv -F $'%T %t %r %e\n' ?
    – ilkkachu
    Sep 22, 2021 at 17:44
  • Ok great, it works fine, print new line for each update. But why I cannot pipe it through and use awk, grep or tr commands. I tried redirecting stderr to stdout but no luck. Tried using pipe tr -d ':' but it does not work Sep 22, 2021 at 19:05

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .