Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I wrote a little bash script using sed on some html pages to extract some urls.

To avoid each time grabbing sed results in a variable then read it again I simply made 3 functions and piped together.

first_function $1 | second_function | third_function

Let's say that:

  • the first function finds item urls in a list from a given href ($1)
  • the second function extracts from each of the piped urls an image src
  • the third function assembles output HTML

at the moment I echo $lot_url in second_function so I can read it in third_function along with $img_url and put it in HTML.

It would be cleaner if I could just hold it as a global variable that's accessible from second_function to third_function but seems I can not.

This becomes more necessary as the number of values to pass between functions grows.

Here a full sample code:

first_function(){
    curl -s "$1" | sed -nr '
        #extract sub urls
    '
}

second_function(){
    while read lot_url; do
        echo "$lot_url"
        curl -s "$lot_url" | sed -nr '
            #extract img src
        '
    done
}

third_function(){
    while read lot_url; read img_url; do
        echo "<a href="$lot_url"><img src="$img_url" /></a>"
    done
}

first_function "$1" | second_function | third_function
share|improve this question
add comment

1 Answer

up vote 2 down vote accepted

The two sides of a pipe are in different processes. You can't share variables between these processes. If you want to share data, you either have to pass it through the pipe, or use alternate communication channels. If you need alternate communication channels, you're above the shell's capabilities, switch to a real programming language.

Here, passing lot_url alongside img_url in the second pipe seems like a good solution to me. I'd pass them on the same line. Assuming your URLs are properly escaped, you don't need any particular quoting, you can pass them on the same line. This would have the advantage of allowing a variable number of img_urls in each lot_url.

second_function(){
    while read lot_url; do
        echo "$lot_url"
        curl -s "$lot_url" | sed -nr -e '
            #extract img src
        ' -e "s>^>$lot_url >"
    done
}

third_function(){
    while read lot_url img_url; do
        echo "<a href="$lot_url"><img src="$img_url" /></a>"
    done
}
share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.