3

I'm performing several commands on large files from within a bash script. In order to monitor the progress I use the pv command.

An example command could look like this

cat $IN_FILE | pv -w 20 -s $(du -sb $IN_FILE | awk '{print $1}') | grep ...

The script contains multiple commands of similar structure, rather than having to use the same prefix on all of those commands over and over again I defined a function for the progress monitoring part as short hand. (Note: the fSize and fSize Sum functions wrap the du command part from the above example, where fSizeSum returns the sum of the size of two files fSize the size of one file):

function prog()
{
    local __size
    if [[ $2 ]]
    then
        __size=$(fSizeSum $1 $2)
    else
        __size=$(fSize $1)
    fi

    echo "cat $1 | pv -w 20 -s ${__size}"
}

Actually that approach already shortens the commands down a bit. I can execute the above example now like this:

eval "$(prog $IN_FILE) | grep ..."

What is still bothering me, is that I now have to quote each line and use eval. I'd rather like to use the function "prog" like it was a regular shell command i.e.:

prog $IN_FILE | grep ...

All my attempts to modify my function to work as a prefix in a chain of piped commands have failed so far.

Is there any way I can make self defined bash-script function to act like it was a regular bash command and put it in front of a pipe chain? If there is no way to do so using functions is there any other method to achieve this goal - like defining a local alias within the script?

  • I must be misunderstanding your question, but what happens if you replace echo "cat $1 | pv -w 20 -s ${__size}" with just cat $1 | pv -w 20 -s ${__size} in your function definition? – dhag May 13 '15 at 19:11
  • Nope you're not misunderstanding it - removing echo and quotes actually does the trick. I actually can't believe it was so easy. The quoting and bracing in bash gets me every time. If you post your comment as answer I'll mark it as correct and give you a vote. – norritt May 13 '15 at 19:28
2

In your function definition, I would suggest replacing:

echo "cat $1 | pv -w 20 -s ${__size}"

with just:

cat $1 | pv -w 20 -s ${__size}

This way, the function itself will execute this bit of code, without requiring a call to eval in the caller.

  • Works perfectly, thanks for the quick reply. Unfortunately I can't give you an upvote yet, since that apparently requires at least 15 reputation points. – norritt May 13 '15 at 19:47
1

pv behaves as cat if you pass files as arguments, and in this way you don't even need to pass -s.

So you could just do:

pv -w 20 $IN_FILE | grep ...

  • Yes nice improvement, works perfectly and might even result in a little performance plus, thanks for the hint! – norritt May 14 '15 at 9:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.