I've created a script that uses a for loop to loop through an IBM script to get the size of directories. The script then outputs the directory size and path to a Slack channel for easy viewing. The program works but the output of the IBM script is large and needs formatting to make it easy to read in Slack. So I need to collect two pieces of information which will be piped into two separate variables with the variables being used to create the Slack message. My script looks like:

SIZE () {
  for dir in /path/to/dir/*
    cd /usr/lpp/mmfs/samples/ilm/
    SLACKMESSAGE=$(./mmpolicy-du.sample "$dir" -t /mmfs1/.policytmp -g /mmfs1/.policytmp/ -N all -v -h)

So /path/to/dir/* is the parent directory and the IBM script ./mmpolicy-du.sample "$dir" -t /mmfs1/.policytmp -g /mmfs1/.policytmp/ -N all -v -h will loop through all the child direcotries (one level deep) and get their size. The output looks like:

[I] 2018-05-31@16:32:55.798 Policy execution. 0 files dispatched.
[I] 2018-05-31@16:32:55.804 Policy execution. 0 files dispatched.
File system scan complete.
534.5M     total
mmapplypolicy du for /path/to/directory/SPI/ complete at Thu May 31 17:32:55 2018

That's not the cleanest output so I'd like to pipe the directory size in the above example case 534.5M into a variable called SIZE and the SPI into another variable called PROJECT. Of course being a loop the SIZE and PROJECT variable will change on each directory inside the above example of /path/to/dir/. The Slack function which gets called in my above function will use those two variables. I'm struggling to work out how to get those two into variables. Anyone have a solution? Thanks!

closed as unclear what you're asking by Jesse_b, Rui F Ribeiro, Romeo Ninov, roaima, Timothy Martin Jun 1 '18 at 19:23

Please clarify your specific problem or add additional details to highlight exactly what you need. As it's currently written, it’s hard to tell exactly what you're asking. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

  • 3
    Show us an example input and a desired output. – Arkadiusz Drabczyk May 31 '18 at 16:48
  • 2
    Looks like an XY problem. What do you want to achieve in the end? – Dani_l May 31 '18 at 17:09
  • 1
    I think this is still unclear. It may be better to explain what your end goal is. You say this program will have lots of output and you want to assign A variable for each. Do you need a new variable for each or reuse the same one to perform a task? How do you want to accomplish this? With a script that wraps around your program or are you expecting to just run the program through a pipeline? – Jesse_b May 31 '18 at 17:10
  • @Dani_l Yes Sorry I meant PROJECT as the variable. Question edited. – sark May 31 '18 at 18:22
  • Some sample data here (input and desired output) would be really useful. – roaima Jun 1 '18 at 17:12

To do it in one go:

eval "$(yourscript | awk -v q=\' '
  $1 ~ /^[[:digit:]]+(\.[[:digit:]]+)?[MGT]$/ {
    print "SIZE=" $1
  match($0, /[[:upper:]]{3}/) {
    print "PROJECT="q substr($0, RSTART, RLENGTH) q

Note that some awk implementations like the nawk or Solaris, mawk or older versions of gawk don't support the {x,y}/{x} regular expression interval operators (in the case of older (and not ancient) versions of gawk, you can run it with POSIXLY_CORRECT=anything in the environment for it to support them though) even though it has been standard for over 25 years now. With those, you would need to replace [[:upper:]]{3} with [[:upper:]][[:upper:]][[:upper:]].

  • I like this @StéphaneChazelas. I edited my function but when I ran it the size was returned but not the 3 capital letters. – sark May 31 '18 at 20:55
  • @sark, possibly your awk is one of those implementations that don't support interval RE operators. Try replacing [[:upper:]]{3} with [[:upper:]][[:upper:]][[:upper:]] – Stéphane Chazelas May 31 '18 at 23:09

I doubt very much this is what you need, but this answers you current question verbatim.

First part (assuming GNU grep or compatible):

project="$(yourscript|grep -oE '[[:upper:]]{3}')"

Second part:

size="$(yourscript|grep -oE '[[:digit:]]+(\.[[:digit:]]+)?[MGT]')"

Combined it's simpler to run the script once, save the output and grep and assign on the saved output:

size="$(printf '%s\n' "$output"|grep -oE '[[:digit:]]+(\.[[:digit:]]+)?[MGT]')"
project="$(printf '%s\n' "$output"|grep -oE '[[:upper:]]{3}')"

Some explanation
grep -o - only return the object grepped, not the entire line
-E - Extended regex
[[:upper:]] - match uppercase letters only ([A-Z], but regardless of locale)
{3} - limit the match to exactly 3 consecutive chars.
[[:digit:]] - match digits ([0-9], again w/o regard to locale)
+ - match 1 or more times
\. - match a dot
(...)? - match 0 or 1 times - this ensures numbers without a decimal dot might also be captured
[xy] - matched exactly one of the items x or y.

  • @Jesse_b thx, corrected – Dani_l May 31 '18 at 17:49
  • And quote the shell globs and avoid echo for arbitrary data. [x|y] matches on x, | or y. You need [xy] for either x or y. – Stéphane Chazelas May 31 '18 at 17:50
  • @StéphaneChazelas can you show what you mean ? I'm not sure what the problem with echo or current quoting is – Dani_l May 31 '18 at 17:56
  • See edit and Why is printf better than echo? – Stéphane Chazelas May 31 '18 at 23:12
  • @StéphaneChazelas Thank you, it's much clearer now. – Dani_l Jun 1 '18 at 15:38

Not the answer you're looking for? Browse other questions tagged or ask your own question.