The tag has no usage guidance.

learn more… | top users | synonyms

38
votes
2answers
2k views

How do I distribute a large download over multiple computers?

I need to download a large file (1GB). I also have access to multiple computers running Linux, but each is limited to a 50kB/s download speed by an admin policy. How do I distribute downloading this ...
53
votes
4answers
28k views

Throttle the download speed of wget or curl while downloading

Is it possible to throttle (limit) the download speed of wget or curl ? Is it possible to change the throttle value while it is downloading ?
11
votes
3answers
13k views

wget - How to download recursively and only specific mime-types/extensions (i.e. text only)

How to download a full website, but ignoring all binary files. wget has this functionality using the -r flag but it downloads everything and some websites are just too much for a low-resources ...
4
votes
2answers
3k views

Problem with recursive download using wget

I am trying to learn how to use recursive download with wget from the wget info page. For example, let us try to download all the images of xkcd. A list of all the pages is present in the xkcd ...
2
votes
3answers
65 views

Accessing each line in .txt file, download and rename links

Let's say I have a .txt file where I have a list of image links that I want to download. example: image.jpg image2.jpg image3.jpg I use: cat images.txt | xargs wget and it works just fine What I ...
6
votes
3answers
16k views

How to call a service URL from bash shell script in parallel?

I have a service which I am calling from another application. Below is my service URL which I am calling - http://www.betaservice.domain.host.com/web/hasChanged?ver=0 I need to do some load test ...
3
votes
0answers
65 views

Use command line to download file which is accessible only in a given session [duplicate]

Possible Duplicate: Does anybody here have experience in automating some tasks in web applications using curl? Sorry for the lengthy title. I am trying use command line to download a file,...
3
votes
1answer
1k views

How to download all html files from a URL's directory with wget?

I'd like to use wget to download a tutorial in a website, but this tutorial is in a specific directory. Suppose the URL is the tutorial main page http://www.abc.com/aaa/bbb/ccc/index.html. But the ...
2
votes
1answer
4k views

ffmpeg and libmp3lame produces bad audio quality?

when I get a flash video from YouTube, why is the quality of the audio much worse than the origin video on YouTube? When I downloaded the flash movie, I convert it to avi like this: ffmpeg -i ~/"$...
1
vote
1answer
207 views

Alert when a process exits

I often download files using wget and cURL. Is there anyway I can make my Linux box give a visual alert when either of those processes exit (i.e: the download completes) PS : I am using Ubuntu ...
1
vote
1answer
663 views

How do I create a local copy of a complete website section from OSX using curl?

I'm trying to download Mathematics of the Discrete Fourier Transform so that I can read through it off-line on my iPad (which doesn't have an Internet connection) I have used wget in Linux. It seems ...
1
vote
2answers
474 views

How do I download PDFs from PubMed, given their IDs?

I have PubMed IDs for files I need to download from the PubMed site in .pdf format. I tried this script: #!/usr/bin/env bash Link="http://www.ncbi.nlm.nih.gov/pubmed/" ID=(10021369 ...
0
votes
1answer
195 views

Console download accelerator that downloads *multiple* files simultaneously

The download accelerators I've found thus far only let me specify a single file to download. This is useful for single large files, but I'm looking for a tool that lets me hand over a list of multiple ...
-3
votes
2answers
4k views

How should I download Red Hat Enterprise Linux? [closed]

I am aware that Red Hat Enterprise Linux (RHEL) is not free, that is, Red Hat charges money for the support it gives for RHEL, and I am also aware of the Fedora project of Red Hat. But I want to ...