4

I'm trying to download several files with wget. The relevant URLs are stored in the file url.txt. When using the command wget --content-disposition -i url.txt it doesn't find the .txt file.

Where do I have to store the url.txt file?

  • Does it give you any indications of a error msg? – slm Aug 27 '14 at 9:02
  • @Chris: In that case please check if that path exists or not.And also check for the read permission of that file.By looking at your command that is wget --content-disposition -i url.txt you should execute this command in the directory where your url.txt is present.Because you are not giving an absolute path to wget. – Thushi Aug 27 '14 at 9:09
  • if you use that syntax the url.txt needs to be in the same directory in which you run wget – pqnet Aug 27 '14 at 9:13
  • @ Thushi: how would the command look like if I would give an absolute path? – Chris Aug 27 '14 at 9:25
  • @Chris : Hmmm...It's a complete path like wget --content-disposition -i /home/Chris/Samples/url.txt – Thushi Aug 27 '14 at 9:38
5

You should give your URLs with http://

Your url.txt should have

  http://www.google.com
  http://www.yahoo.com

If you don't have the protocol included you'll get messages stating the following:

$ wget --content-disposition  -i url.txt 
url.txt: Invalid URL www.google.com: Scheme missing
url.txt: Invalid URL www.yahoo.com: Scheme missing
  • @ Thushi: to put it in other words, what is the most suitable command to access the url.txt file to download the data? – Chris Aug 27 '14 at 9:19
  • @Chris: It is a different question.May be you should ask another question for this.You can use wget curl and some other utilities. – Thushi Aug 27 '14 at 9:26
  • If this is the problem, it would be trivial to fix the file with sed, ed, vim, or any other editor with regular expressions. Something like: sed -i '/^http:\/\//!s/^/http:\/\//' url.txt. – evilsoup Aug 27 '14 at 10:50
0

Please make sure that file should be in .html format containing URL please refer wget man page . -i file --input-file=file Read URLs from file, in which case no URLs need to be on the command line. If there are URLs both on the command line and in an input file, those on the command lines will be the first ones to be retrieved. The file need not be an HTML document (but no harm if it is)---it is enough if the URLs are just listed sequentially. However, if you specify --force-html, the document will be regarded as html. In that case you may have problems with relative links, which you can solve either by adding "" to the documents or by specifying --base=url on the command line.

Thanks

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for?Browse other questions tagged or ask your own question.