Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I'm trying to automate a curl script and eventually make it loop also. I found that I can't use any loop like:

for i in `cat authors`
do
    curl *line here*   "www.test.com/autors?string="$i"&proc=39"
etc

or even env glob variables for CURL

Is there a way to loop a get request on curl for one variable?

curl -H "Host: www.test.com" -G "www.test.com/autors?string="author+name"&proc=39"

All I get is the traffic graph CURL has and no good result. I'm using bash

share|improve this question
1  
Yes, it is doable. But it is very difficult to answer without more details. What does authors contain, and what output would you like from curl? –  Anders Lindahl May 11 '11 at 18:34
    
authors contain names like with spaces and i guess dots –  CurtyJB May 11 '11 at 18:40
2  
Note: you spelled 'authors' as 'autors' in your code example. That isn't throwing your program off is it? –  Caleb May 11 '11 at 18:58
add comment

2 Answers

Try this construct instead:

 while read author; do
      curl "http://www.test.com/authors?string=$author&proc=39"
 done < authors

This will save you the cat, having to do things in backticks, etc. It would also allow you to read several columns out of the input file if you wanted to, but specifying just one variable to fetch like in my example will get the whole line.

share|improve this answer
    
i dont think this will work –  CurtyJB May 11 '11 at 18:46
2  
How about you try it and say what doesn't work specifically. This structure makes it a lot easier to handle spaces and such as the whole line gets read into the variable. –  Caleb May 11 '11 at 18:53
2  
If ${author} includes spaces or similar characters, you will want to URL encode them, otherwise you may find you get errors like code 400, message Bad request syntax ('GET /authors?string=shakespeare, w.&proc=39 HTTP/1.1') from your HTTP server. –  Johnsyweb May 11 '11 at 19:20
add comment

authors contain names like with spaces and i guess dots

Several things going on:

First of all, the way you're doing the for loop simply doesn't work. Try this:

 for i in `cat authors`
 do
    echo "Author = \"$i\""
 done

See what is going on? Every white space separates a value for $i. If you have first and last names, you'll be getting one line with just a first name and one line with just a last name.

Now try this:

cat authors | while read i
do
    echo "Author = \"$i\""
done

A bit better? Now, $i is your entire line. (Yes, I know about the "useless cat", but see where I'm going here!)

Second of all, you may have to convert those spaces and periods into valid URL characters. Spaces must be converted into %20. Let's take a different approach:

 sed 's/ /%20/g' authors | while read test
do
    echo "curl *line here* www.test.com/authors?string=$i&proc=39"
done

(Now, you see why I had the cat.)

This should print out your curl command. What I am doing is taking the authors file and converting the spaces to the string %20. Now, I can get rid of the quotes around the string which are also causing problems.

Thus we can now get rid of the echo, and run the actual CURL command:

 sed 's/ /%20/g' authors | while read test
do
    curl *line here* www.test.com/authors?string=$i&proc=39
done

Now what does the *line here* represent? Is that your list of options? Oh, and you had autors instead of authors in your program. Did you mistype it here, or is that the way your program suppose to be?

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.