0

I have two commands, build and deploy. Currently I run build manually, parse its output using my own eyes, and use a value I find in the output as an argument to deploy. The process looks something like:

$ build
==> amazon-ebs: amazon-ebs output will be in this color.
... hundreds of lines of output ...
==> Builds finished. The artifacts of successful builds are:
--> amazon-ebs: AMIs were created:
us-east-1: ami-19601070

$ deploy ami-19601070
... some more output ...

(build is actually Packer, for the astute)

I would like to tie these two steps together in a script. The rough outline would include the following:

  • Run build
  • Make sure the exit status was 0 and that the output contained the string "AMIs were created", otherwise abort
  • Extract the AMI number (ami-19601070) from the output
  • Run deploy ami-19601070

I'm trying to come up with the best way to connect everything together, ideally by using a shell script, but I'm stuck on how to grep the output for two separate patterns while, ideally, still streaming all stdout/stderr messages to the terminal as the commands run. I'm wondering if I should abandon the idea of doing this in a shell script and instead write a small Python script to do it.

5

Sounds like a job for tee:

build | tee /some/file
ami_name=$(do_something_to /some/file)
deploy "$ami_name"
2
 deployArgs=`build | tee /dev/tty  | extractDeployArgs` &&
    deploy "$deployArgs" #won't run unless extractDeployArgs suceeded

tee /dev/tty will print directly to the terminal and pass the output to the next command in the pipeline at the same time.

(Feel free to replace it with some other file (or /dev/fd/"$someFileDescriptor" if you need want the side output to go to $someFileDescriptor))

In more advanced shells (ksh, bash, zsh, but not in dash) you can set -o pipefail to make sure the pipeline fails if any of its links fails (useful if extractDeployArgs can't tell from its input whether build suceeded or not).

  • 1
    xargs deploy has quite inane rules for input delimitation. I'd suggest avoiding it, or using -0 instead. Also, going out to /dev/tty results in some potentially undesirable properties (such as the output defying further redirection) -- I'd suggest switching around FDs instead if you want to go this route. – Chris Down Nov 24 '15 at 17:46
  • @ChrisDown Thanks for the commentary. I improved/simplified it a little, taking your feedback into account. My main point was that saving to a file isn't necessary. – PSkocik Nov 24 '15 at 18:39
  • 1
    @Chris Down - quoting aside, xargs -t kind of does exactly what is asked for here. -p is even closer. – mikeserv Nov 25 '15 at 11:39
0
tf=$(mktemp)
build | tee "$tf"
grep -Fq 'AMIs were created' "$tf" && ami=$(grep -o 'ami-[0-9]\+$' "$tf")

# you didn't say if `deploy` can handle multiple args or not.
# uncomment one of the following:
# deploy $ami
# for a in $ami ; do deploy "$a" ; done
0

If your interest is in a particular string of output you could run it like:

msg="$(build | grep -e "AMIs were created\|ami-[0-9]*")"

if [ -n "$(echo $msg | grep -e "AMIs were created")" ];then
    ami="$(echo "$msg" | grep -e "ami-[0-9]*" | cut -d ' ' -f 2)"
    deploy "$ami"
else
    exit 1
fi

The first grep selects only the lines in your output that report AMI's were created or ami-#. The output is checked for "AMIs were created", and if present, parses the ami-# out of your relevant output and uses it to deploy.

  • Nice idea, but the build output will not be displayed "in real time", which is a requirement. – glenn jackman Nov 24 '15 at 18:14
  • 1
    You can change your if to: if grep -Fq "AMIs were created" <<<"$msg"; then – glenn jackman Nov 24 '15 at 18:15
  • 1
    And quote your variables: unix.stackexchange.com/q/171346/4667 – glenn jackman Nov 24 '15 at 18:23
  • @glennjackman I didn't realize seeing all the output was a requirement, now that I re-read I see it towards the end. I'll leave what I have here in case it might be helpful to the asker for parsing the output. – Centimane Nov 25 '15 at 11:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.