6
votes
3answers
433 views

How you stop 'wget' after it gets a 404?

If you use brace expansion with wget, you can fetch sequentially-numbered images with ease: $ wget 'http://www.iqandreas.com/sample-images/100-100-color/'{90..110}'.jpg' It fetches the first 10 ...
1
vote
1answer
496 views

wget a file, logging the output and showing the output on prompt

In a bash script, I need to download a file from the web, for doing it i use wget command. I would log the output of wget command, and at "the same time" having the output prompting on terminal. I ...
1
vote
2answers
1k views

Type URL and show HTTP status

I want to make a shell script where I type an url and it returns me the status code.So far I've tested wget is there any other cause it has the tendency to save whatever you run. Then I tried to some ...
6
votes
3answers
108 views

generate links of pages with incrementing variable

consider the links http://unix.stackexchange.com/questions/tagged/linux?page=2&sort=newest&pagesize=15 ...
2
votes
1answer
686 views

download pdf files from using curl

How to download hundereds of .pdf files from http://www.ncbi.nlm.nih.gov/pmc/articles using a loop, for example for the following document ids: PMC3386155 PMC3625956 PMC3477654 PMC3531051 ...
1
vote
1answer
469 views

How to figure out where a link gets redirected

I currently working on a script that updates a file. The file has several versions. The important ones have static links. These get redirected to the actual file which then gets downloaded using wget. ...
4
votes
2answers
391 views

Download from dynamically changing urls via script

I want to write a script to download all Walter Lewin lectures from this page in wmv format: http://videolectures.net/mit801f99_physics_classical_mechanics/ However the links to the wmv file for ...
2
votes
1answer
524 views

Predict filename before downloading from a URL, in shell script

I have a shell script that downloads files from a list using wget, and resumes automatically if there is any non-critical error (because of unstable WIFI during storms). The problem is, I want to ...
3
votes
1answer
397 views

Extract directory from wget's stdout

I am trying to wget a tarball from github.com and, without creating a temporary file, extract a subdirectory from it: wget -qO- https://github.com/django-nonrel/django-nonrel/tarball/develop | tar ...