I have the following shell-script from cleaning output of a script so it's descending, gives package names and cleanly exists -

┌─[shirish@debian] - [~] - [6348]
└─[$] cat find-interesting-patches.sh 

for source in $(dpkg-query --show -f \
 '${source:Package}\n' | sort -u); do bts \
 select source:${source} tag:patch \
| sed  "s/^/${source} /g"; done 

and I get the output on stdout.

┌─[shirish@debian] - [~] - [6349]

2048-qt 781691
abootimg 725728
accountsservice 805989
acl 677570
acpi 772688
acpi-support 823072
acr 772194

Now I know that the number of packages is finite, for instance it is 4k and a little bit more from doing

$dpkg -l > dpkg-l.txt

and then

$wc dpkg-l.txt 

which gives line numbers, nummber of words the whole she-bang.

When I run the script is there a way to have a progress bar so I know how much of it has been completed. I looked at https://stackoverflow.com/questions/238073/how-to-add-a-progress-bar-to-a-shell-script#238094 and tried some of the things but they didn't work. For instance I tried pv and it failed spectacularly -

┌─[shirish@debian] - [~] - [6353]
└─[$] pv ./find-interesting-patches.sh

for source in $(dpkg-query --show -f \
 '${source:Package}\n' | sort -u); do bts \
 select source:${source} tag:patch \
| sed  "s/^/${source} /g"; done 
 153 B 0:00:00 [9.09KiB/s] [=================================================================================================>] 100% 

Not what I was expecting, does anybody know what am I doing wrong ?

  • pv measures the amount of data passing through it. Your script doesn't write anything until the last sort finishes, which makes pv useless. There is no simple way to achieve what you want with the script you link to, you'll have to re-think the script from scratch. – Satō Katsura Nov 24 '16 at 7:41
  • actually the script I am doing doesn't do the sort bit at all. I'm doing that manually for the same reason as you shared. I've shared the script above, if you look at the output of $ pv ./find-interesting-patches.sh you would see that the second sort is not used at all. – shirish Nov 24 '16 at 7:44
  • @SatoKatsura see above ^ – shirish Nov 24 '16 at 7:50
  • The point is the data you're feeding to pv comes too late to be of any use, regardless of why that happens. – Satō Katsura Nov 24 '16 at 7:55

When you give pv a file as command-line argument, it acts like cat and displays the file, and tracks progress of that output, which is what you're seeing. That form is useful if you're using pv to feed a file to a slow process (using a pipe); the manpage has examples.

You might think of saving the output of

dpkg-query --show -f '${source:Package}\n' | sort -u

to a file and using pv to feed that to a loop calling bts select, but the resulting file is so short that it gets buffered and pv considers it's been processed before processing actually begins.

So you need to track progress manually:

dpkg-query --show -f '${source:Package}\n' | sort -u > ${file}
lines=$(wc -l ${file} | cut -f1 -d\ )
while read source; do
    line=$((line + 1))
    printf "Processing line %d / %d\r" ${line} ${lines} >&2
    bts select source:${source} tag:patch | sed "s/^/${source} /g"
done < ${file} | sort -k2,2n -r
rm ${file}
echo "Done                           " >&2

I'll let you adapt that to use a nice progress bar, using the examples in How to add a progress bar to a shell script?.


If you time the command, you can use this progress bar

It goes smooth by using half block characters


enter image description here

Another option that is not always available is to update the progress bar not based on time, but at certain points of your script.

code set_progress 10 code set_progress 20 ... code set_progress 100

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.