1

I want to create a script that will let me download images from entire subreddits at once, but I'm currently stuck on just getting the links right (one problem at a time! ^^). Anyway, since I can't think of a way to determine what format the image in the link will be, I was hoping sed could be used to specify multiple extensions during replacement (e.g. (jpg|png|gif)), but this does not seem to work.

grep -E "href=\"/r/bacon/*" bacon.htm | sed "s .*/r/bacon/\(.*\)\".* http://i.imgur.com/\1.\(\jpg\|\png\|\gif\) "

expected:

http://i.imgur.com/foo.jpg
http://i.imgur.com/foo.png
http://i.imgur.com/foo.gif

got:

http://i.imgur.com/foo.(jpg|png|gif)

Google also does not seem to have an answer to this, admittedly, weird problem or at least I am having trouble coming up with meaningful ways to describe it.

Any ideas or should I try this from a different angle?

  • Your question is unclear. You may want to show a more step-wise statement of the problem. – unxnut Feb 20 '14 at 4:11
  • Sorry about that, but like I said, the problem is weird and difficult to describe (at least to me). My question is: How can one edit a string in such a way (using sed) that the output produces multiple variations of that string? I think TimP provided a good solution to this question, but I do find drewbenn's solution to the overall problem more elegant. – muser Feb 20 '14 at 22:52
1

If I understand your question right your grep is going to produce a bunch of strings like this:

href="http://reddit.com/r/bacon/foo"

And you want to turn each of them into something like:

http://i.imgur.com/foo.jpg
http://i.imgur.com/foo.png
http://i.imgur.com/foo.gif

It's not particularly graceful, but you could just do:

sed "s .*/r/bacon/\(.*\)\".* http://i.imgur.com/\1.jpg\nhttp://i.imgur.com/\1.png\nhttp://i.imgur.com/\1.gif "

Example:

timp@helez:~/tmp$ cat bacon.lines 
href="http://reddit.com/r/bacon/foo"
href="http://reddit.com/r/bacon/bar"
timp@helez:~/tmp$ cat bacon.lines | sed "s .*/r/bacon/\(.*\)\".* http://i.imgur.com/\1.jpg\nhttp://i.imgur.com/\1.png\nhttp://i.imgur.com/\1.gif "
http://i.imgur.com/foo.jpg
http://i.imgur.com/foo.png
http://i.imgur.com/foo.gif
http://i.imgur.com/bar.jpg
http://i.imgur.com/bar.png
http://i.imgur.com/bar.gif
1

Put it inside a loop, instead

extension_list="jpg png gif" #easy to add other extensions later, like JPG
: # ... other stuff in the script
for extension in $extension_list
do
    sed "s/foo\(bar\)baz/\1.$extension/"
    # if you get the file you want, you can use 'break' to exit the loop early
done

This is out of scope of your actual question, but instead of combining grep and sed you can (if you want) use just sed to print matching lines, like:

sed -n "s+.*/r/bacon/\(.*\)\".*+http://i.imgur.com/\1.$extension+p" bacon.htm
  • Cool solution, much more elegant than my approach. Just out of curiosity, how can I use 'break' to exit the loop early? Could you provide a quick example of this? – muser Feb 20 '14 at 23:02
  • @muser x=1; while [[ $x -lt 10 ]]; do x=$((x+1)); echo $x; if [[ $x -gt 3 ]]; then break; fi; done. In your case, you would probably try fetching the file inside the loop; if you get a good file, then break instead of trying the rest of the extensions (since they would probably fail, anyway). – drewbenn Feb 21 '14 at 0:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.