I was trying to create a function in my bash profile that would pass in a variable (mainly a directory), and run two different programs. The variable would typically be something like ~/logdirectory/*.log. I've got ~25 files in the /logdirectory/. When I run each line of the function on the command line, it functions as expected.

The first line runs a python script, which on the command line will return the results for all 25 files, but when I run it within the function, it only returns the results for 1 file (the very first in the directory)

The second line is supposed to grep through that same set of log files in /logsdirectory/, and look for a value that should be in all of the log files, and then pipe all of the results into awk to do some maths on the returning values. However, much like the first line, it's only returning the results for the very first file in the directory. Does anyone know what I'm missing or might need to do? I've tried subfunctions with for/do loops to try and get it to work, but no luck. Below is the function I'm working with. Any guidance would greatly be appreciated. Thanks!

plotstats() {
    #take log directory and run plotman analyze
    plotman analyze $1
    grep -i "total time" $1 |awk '{sum=sum+$4} {avg=sum/NR} {tday=86400/avg*6*101.366/1024} END {printf "%d K32 plots, avg %0.1f seconds, %0.2f TiB/day \n", NR, avg, tday}' 

Use "$@" instead of just $1. And remember to double-quote your variables (and shell positional parameters - e.g. use "$1" instead of unquoted $1) when you use them.

plotstats() {
    #take log directory and run plotman analyze
    plotman analyze "$@"
    grep -h -i "total time" "$@" |
      awk '{sum=sum+$4}
           END {printf "%d K32 plots, avg %0.1f seconds, %0.2f TiB/day \n", NR, avg, tday}' 

BTW, the three {} blocks in your awk script can be merged into just one, separated by newlines or semi-colons. There's no conditional, they're always going to run for every input line...so there's no need to have them as separate blocks.

I don't know what plotman does, so I'm assuming it can handle multiple filename args on the command line. If it can only take one filename at a time, wrap it in a for loop instead of using "$@" for this command. e.g.

for f in "$@" ; do
  plotman analyze "$f"
  • note: depending on what you want to do with the output of grep, you will sometimes need to use the -h (aka --no-filename) so that grep doesn't print the filename with each match. Your awk script here only uses $4, not $1 and doesn't use : as a field delimiter, so it's probably OK without -h. unless there are filenames with spaces or tabs in them, that would mess up awk's field numbering. I'll add the -h in my answer.
    – cas
    Apr 29 at 15:58
  • The reason this is the correct answer is because, when you enter plotstats /directory/*.log -- the shell expands the glob pattern first, before invoking the function. Therefore, the function is called with the 25 files, not just that single pattern. Apr 29 at 22:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.