The tag has no wiki summary.

learn more… | top users | synonyms

33
votes
1answer
2k views

How do I distribute a large download over multiple computers?

I need to download a large file (1GB). I also have access to multiple computers running Linux, but each is limited to a 50kB/s download speed by an admin policy. How do I distribute downloading this ...
23
votes
4answers
10k views

Throttle the download speed of wget or curl while downloading

Is it possible to throttle (limit) the download speed of wget or curl ? Is it possible to change the throttle value while it is downloading ?
1
vote
1answer
3k views

How to download all images from a website (not webpage) using the terminal?

I want a command that I type a URL, for example photos.tumblr.com, and it download all photos on this site in a folder, but not only images of the site's homepage. This command need to download the ...
5
votes
3answers
5k views

wget - How to download recursively and only specific mime-types/extensions (i.e. text only)

How to download a full website, but ignoring all binary files. wget has this functionality using the -r flag but it downloads everything and some websites are just too much for a low-resources ...
3
votes
1answer
1k views

Problem with recursive download using wget

I am trying to learn how to use recursive download with wget from the wget info page. For example, let us try to download all the images of xkcd. A list of all the pages is present in the xkcd ...
4
votes
3answers
1k views

How to call a service URL from bash shell script in parallel?

I have a service which I am calling from another application. Below is my service URL which I am calling - http://www.betaservice.domain.host.com/web/hasChanged?ver=0 I need to do some load test ...
3
votes
1answer
886 views

How to download all html files from a URL's directory with wget?

I'd like to use wget to download a tutorial in a website, but this tutorial is in a specific directory. Suppose the URL is the tutorial main page http://www.abc.com/aaa/bbb/ccc/index.html. But the ...
3
votes
0answers
64 views

Use command line to download file which is accessible only in a given session [duplicate]

Possible Duplicate: Does anybody here have experience in automating some tasks in web applications using curl? Sorry for the lengthy title. I am trying use command line to download a ...
2
votes
1answer
4k views

ffmpeg and libmp3lame produces bad audio quality?

when I get a flash video from YouTube, why is the quality of the audio much worse than the origin video on YouTube? When I downloaded the flash movie, I convert it to avi like this: ffmpeg -i ...
1
vote
1answer
620 views

How do I create a local copy of a complete website section from OSX using curl?

I'm trying to download Mathematics of the Discrete Fourier Transform so that I can read through it off-line on my iPad (which doesn't have an Internet connection) I have used wget in Linux. It seems ...
1
vote
1answer
148 views

Alert when a process exits

I often download files using wget and cURL. Is there anyway I can make my Linux box give a visual alert when either of those processes exit (i.e: the download completes) PS : I am using Ubuntu ...
0
votes
1answer
106 views

Console download accelerator that downloads *multiple* files simultaneously

The download accelerators I've found thus far only let me specify a single file to download. This is useful for single large files, but I'm looking for a tool that lets me hand over a list of multiple ...