Tag Info

New answers tagged

0

I had a similar case where I did the equivalent of what would be for you --exclude-directories '/tmp' and it worked for me. See also https://serverfault.com/questions/156045/how-to-download-with-wget-without-following-links-with-parameters/156057#156057 for more powerful options on recent wget (regular expressions).


0

you can try, if you are on firefox, installing cliget addon. For every file that you want to download program show you curl, wget or aria2 direct link for downloading that file. I use that to download files on other machines via ssh because it's much faster for me to download directly to remote computer then to upload that file from my computer. This also ...


0

use spider option and pipe the output to grep to get only the links out of them wget --spider --force-html -r -l2 "www.domain.com" 2>&1 | grep '^--' | awk '{ print $3 }' | grep '\.\(jpeg\|jpg\)$' this will display all the links to the standard output, you direct it to a file: wget --spider --force-html -r -l2 "www.domain.com" 2>&1 | ...


4

Check the file size. You're probably actually getting back HTML. Oracle does not give you the jdk download unless you check the checkbox accepting their terms. (If you look at the reponse headers, you're probably getting back Content: text/html) You can accept the terms by providing the following header: --no-cookies --header "Cookie: ...


0

wget http://download.oracle.com/otn-pub/java/jdk/7u67-b01/jdk-7u67-linux-x64.tar.gz -O This will let you download it as a .tar.gz Look at the section of the manual for -O or --output-file=


0

Look into the file using less and you'll find this: <html> <head> <title>Unauthorized Request</title>


1

Notice how it says [text/html] for the second one, not [text/plain]. You're not fetching the package list, you're fetching a webpage that contains the package list, along with a bunch of other formatting. But really, you want to fetch the compressed version; it will save both you and Debian time and money. You can do it with one command: wget ...


1

You could use wget with recursive downloading: wget -nc -nd -nH -np -r -R '*.*' http://ftp.gnu.org/gnu/bash/bash-4.3-patches/ Explanation: -nc: no-clobber (don't overwrite existing files), probably not necessary. -nd: Don't create hierarchy of directories. -nH: Don't create directory based on hostname. Or you'd find everything downloaded to a directory ...


0

for i in $(seq -f "%03g" 1 999); do wget http://ftp.gnu.org/gnu/bash/bash-4.3-patches/bash43-$i if [[ $? -ne "0" ]]; then MAX=$(expr $i - 1) break; fi done echo $MAX files downloaded


0

Sending and responding to HTTP requests are very different tasks. Most software only handles one of the sides: client-side (sending the request and reading the reply) for web browsers and tools like wget, server-side (listening to requests and replying to them) for web servers. Proxies such as Charles bridge between the two sides. Capturing (i.e. listening ...


0

No, wget is the wrong tool for this sort of thing. However, if you want something like charles that gives you the scriptability and customization of GNU tools, you might want to consider using squid. That's one of the more popular proxies for Linux, and you can customize it pretty heavily.


0

Either it is a 403 page. You are not allowed. Or you can use a user agent such as this. wget --header="User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_0) AppleWebKit/600.1.17 (KHTML, like Gecko) Version/8.0 Safari/600.1.17" http://website.com It will allow you to access some websites. If this doesn't work then your are probably not allowed to ...



Top 50 recent answers are included