Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I currently working on a script that updates a file. The file has several versions. The important ones have static links. These get redirected to the actual file which then gets downloaded using wget. I have figured out that wget has a flag whcih prints the headers recived. There is a list of Locations. The last location meantoned in the header is the actual URL. I need to get that!

My idea was to use wget -S to get the header (I need another flag that prevents the file from downloading and creating). Then use a pipe to parse the lines and catch the last line containing location. I guess this could be realized by using grep -l -i "location:" | tail -l. Then I should be left with a single line that can be easily parsed.

So the command would look something like this:

# The -??? flag is the one that prevents the file from downloading. (I don't know it)
Location=$(wget -S -??? $URL | grep -l -i "location:" | tail -l)

My question what flag do I have to use to not download the file with wget or is there another way/command to accomplish this?

share|improve this question

1 Answer 1

up vote 5 down vote accepted

What you want is a HEAD request, but wget does not support it; curl does.

Your distribution most probably has curl in repositories.

curl -s -I $URL -I | awk '/Location: (.*)/ {print $2}' | tail -n 1

$ URL=http://unix.stackexchange.com/questions/89282/
$ curl -s -I $URL | awk '/Location: (.*)/ {print $2}' | tail -n 1
/questions/89282/how-to-figure-out-where-a-link-gets-redirected
$ _

Here:

  • -s prevents curl from showing a progress bar;
  • -I makes curl issue a HEAD request;
  • -L makes curl follow redirects (thanks @brianstone), you may want or not want to include this, depending on which redirect headers you want to track;
  • the awk script prints the matched expression in parens, just the local part of the URI.
share|improve this answer
    
Ok. But if I now modified the pattern like this: /Location: .*\/([^\/]*)\/.*/ how woul I get the what's in the brakets? –  BrainStone Sep 3 '13 at 17:01
    
If you want to get a particular subexpression, try something this: curl -s -I $URL | awk '/Location: .*/ {where=match($2, "[0-9]+"); print substr($2, where, RLENGTH)}' | tail -n 1. In my example, it prints the 89282. –  9000 Sep 3 '13 at 17:39
    
/* Dear readers, if you want to use -L instead of -I above, please write a comment explaining the benefits of doing so. Just editing my answer is not a good idea. Thanks. */ –  9000 Sep 3 '13 at 18:33
    
@9000, he did, he added another extra line which you forgot to remove when reverting the change. –  Stéphane Chazelas Sep 3 '13 at 20:27
    
Thanks @brianstone, I'm giving you proper credit :) I still wish you did not amend my answer but added a comment below it, so we could discuss if the amendment is appropriate. See, the original post says the the tool does not need to follow redirects, but instead allow to pick a certain Location header. –  9000 Sep 3 '13 at 20:35

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.