Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I'm looking for a way to filter/redirect rsync output in a manner where it can be fed to the "dialog --gauge" command, so I can get a nice looking progressbar during file sync. Currently I have only tested it directly at the prompt, but I'm planning to do this in a (bash) shell script.

I have looked around the internet and found bits and pieces, but I'm still missing something to make it work

(Disclaimer: This might be a totally wrong approach, and is a redirect/piping monstrosity)

What I have currently put together:

rsync -avz --progress -e "ssh" user@server:/home/user/data/ /home/user/data | awk -f /home/user/rsync.awk | sed 's/\([0-9]*\).*/\1/' | dialog --title "My Gauge" --gauge "Hi, this is a gauge widget" 20 70

First I have the actual rsync command with the --progress option Output from rsync is piped into awk and uses the followng awk filter:

{
   if (index($0, "to-check=") > 0)
   {
        split($0, pieces, "to-check=")
        split(pieces[2], term, ")");
        split(term[1], division, "/");
        print (1-(division[1]/division[2]))*100
}
#   else
#   {
#       print "#"$0;
#   }
   fflush();
}

This filters out rsync output and provides the percentage in the following format:

53.7037
55.5556
57.4074
59.2593
61.1111
62.963

So to get rid of the decimal numbers, I feed the output to sed:

sed 's/\([0-9]*\).*/\1/'

Which gives the following output:

64
66
68
70
72
74
75
77

Those numbers are piped into dialog like this:

dialog --title "My Gauge" --gauge "Hi, this is a gauge widget" 20 70 

As far as I know, "dialog --gauge" etc. should accept this, but it just displays progress to be 0% until it suddenly reaches 100%

Can someone point me in the right direction here? Am I far away from a working progressbar? Is there a better way to achieve this?

Regards,

Christer

EDIT: After taking @lynxlynxlynx' answer into account, the correct command line is:

rsync -avz --progress -e "ssh" user@server:/home/user/data/ /home/user/data \ 
 | awk -f /home/user/rsync.awk \ 
 | sed --unbuffered 's/([0-9]*).*/\1/' \ 
 | dialog --title "My Gauge" --gauge "Hi, this is a gauge widget" 20 70
share|improve this question
1  
Sounds like something (probably awk) is buffering the output until the program before it in the pipeline sends the EOF. Dunno how to check or tackle this, though... –  Shadur Aug 6 '12 at 12:12
add comment

1 Answer 1

up vote 9 down vote accepted
for i in 10 20 30; do echo $i; sleep 1; done | dialog --title "My Gauge" --gauge "Hi, this is a gauge widget" 20 70

works fine, so @Shadur is right and there is buffering at play.

Adding the sed stripper into the mix shows it is the culprit (only shows 0 and 30):

for i in 10 20 30; do echo $i; sleep 1; done | sed 's/\([0-9]*\).*/\1/' | dialog --title "My Gauge" --gauge "Hi, this is a gauge widget" 20 70

Now that the problem is known, you have multiple options. The cleanest would be to round/cut the percentage in awk with either math or string manipulation, but since you have GNU sed, just adding -u or --unbuffered should do the trick.

However for completeness' sake, a simple test case shows awk also does buffering:

for i in 10 20 30; do echo $i; sleep 1; done | awk '{print $0}' | sed -u 's/\([0-9]*\).*/\1/' | dialog --title "My Gauge" --gauge "Hi, this is a gauge widget" 20 70 

But you already handle that with fflush, so I don't expect problems.

share|improve this answer
    
Wow..! Thanks a million! Adding --unbuffered to sed was the only thing needed! Thanks for sharing your time and knowledge Shadur&lynxlynxlynx! –  chr1573r Aug 6 '12 at 12:52
    
For other people wondering what the final command looks like: rsync -avz --progress -e "ssh" user@server:/home/user/data/ /home/user/data | awk -f /home/user/rsync.awk | sed --unbuffered 's/([0-9]*).*/\1/' | dialog --title "My Gauge" --gauge "Hi, this is a gauge widget" 20 70 –  chr1573r Aug 6 '12 at 13:10
    
No problem, and welcome to the site. :) –  Shadur Aug 6 '12 at 14:23
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.