Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

CentOS 6.x

I want to take the output from curl, grep for two completely separate strings, and assign their respective values as variables. What is the most efficient way to do this (without writing the output to disk)?

Normally I would think of having a script with something like:

#!/usr/bin/env bash
foo1=$(curl https://domain.com/file.xml | grep string1)
foo2=$(curl https://domain.com/file.xml | grep string2)

But this ends up taking two passes and is horribly inefficient. Is there a better way? Hopefully a solution that involves fewer passes?

share|improve this question

2 Answers 2

up vote 7 down vote accepted

1. Grepping into 1 variable

Try this:

foo1=$(curl https://domain.com/file.xml | grep -E "string1|string2")

This will run curl 1 time and grep for occurrences of string1 or string2.

2. Grepping into 2 variables

If they're different variables then change tactics slightly. Capture the output of curl and then grep afterwards.

output=$(curl https://domain.com/file.xml)
foo1=$(echo "$output" | grep "string1")
foo2=$(echo "$output" | grep "string2")

3. Grepping into an array

You could also store the results in an array instead of separate variables.

output=$(curl https://domain.com/file.xml)
readarray foo < <(echo "$output" | grep "string1|string2")

This one is a little tricky to deal with if your results from the grep might not return results, since the results from "string2" might be the first or second item in the array, but I'm providing it here just as a demonstration of the approach.

4. Reading into vars from grep

Yet another method which makes use of the read command along with process substitution (<( ..cmd..)).

$ read -d"\n" foo1 foo2 \
   <(curl https://domain.com/file.xml | grep -E "string1|string2")

This again can be tricky if the search for "string1" returns nothing, causing any matches for "string2" to show up in $foo1. Also this approach tends to be less portable than either #2 or #3 above.

share|improve this answer
    
Sorry! I had a typo. The variables are actually different. :-( Summary is now updated. –  Mike B Mar 4 at 2:20
    
You might want to note that the process substitution you recommend is far from a portable option. –  mikeserv Mar 4 at 4:13
    
@mikeserv - thanks for the feedback, updated. –  slm Mar 4 at 6:33
read var1 var2 <<CURLSED
$(curl $url | sed -nr 's/(regx1)\|(regx2)/"\1"'"$IFS"'"\2"/p')
CURLSED

Seriously, though, guy, there are like a million ways to skin this cat.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.