1

Is there any way to use common web browsers such as firefox to fetch a page using command line interface? If I use firefox www.google.com, then firefox is opened and shows google page. I want to fetch the web site and save that a one or a bunch of files.

  • 2
    lynx, (e)links(2), w3m text browsers have a -dump option. GUI browsers aren't much for supporting command-line... – Thomas Dickey Jul 4 at 19:43
  • 3
    Most people would use wget or curl, but the question is asking for a web browser. – Thomas Dickey Jul 4 at 22:05
  • Why do you insist on using Firefox, given that it may not be suitable for this task? – Jim L. Jul 4 at 22:07
  • @jim-l: I want to analyze some time related things for fetching web pages. If chrome can do that, I will switch to that. – mahmood Jul 5 at 5:47
0

you can download full webpage using wget:

wget \
     --recursive \
     --no-clobber \
     --page-requisites \
     --html-extension \
     --convert-links \
     --restrict-file-names=windows \
     --domains website.com \
     --no-parent \
         www.website.com

The options are:

  • recursive: download the entire Web site.
  • domains website.com: don't follow links outside website.com.
  • no-parent: don't follow links outside the directory /.
  • page-requisites: get all the elements that compose the page (images, CSS and so on).
  • html-extension: save files with the .html extension.
  • convert-links: convert links so that they work locally, off-line.
  • restrict-file-names=windows: modify filenames so that they will work in Windows as well (if needed).
  • no-clobber: don't overwrite any existing files (used in case the download is interrupted and resumed).

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.