3

I want to download the source files for a webpage which is a database search engine. Using curl I'm only able to download the main html page. I would also like to download all the javascript files, css files, and php files that are linked to the webpage and mentioned in the main html page. Is this possible to do using curl/wget or some other utility?

7

First of all, you should check with the website operator that this an acceptable use of their service. After that, you can do something like this:

wget -pk example.com

-p gets the requisites to view the page (the Javascript, CSS, etc). -k converts the links on the page to those that can be used for local viewing.

From man wget:

-p, --page-requisites

This option causes Wget to download all the files that are necessary to properly display a given HTML page. This includes such things as inlined images, sounds, and referenced stylesheets.

[...]

-k, --convert-links

After the download is complete, convert the links in the document to make them suitable for local viewing. This affects not only the visible hyperlinks, but any part of the document that links to external content, such as embedded images, links to style sheets, hyperlinks to non-HTML content, etc.

  • Would this save all the files in .html .js and .css formats? – user66369 Jun 19 '14 at 8:45
  • @Venkat The page requisites are retrieved in the same format they are offered in. – Chris Down Jun 19 '14 at 8:46
  • I just have one more question. The page I'm downloading is like a database search engine. Does this command download all the php and mysql scripts that might be associated with the webpage? I can only seem to find css, js and html files. Or are the database queries on a different inaccessible file? Also, the webpage has .php extension if that helps. – user66369 Jun 19 '14 at 9:09
  • If they're requisites for the page, they will be downloaded. If they're just linked to from the page and are not page requisites, then you want -R and -A, probably. – Chris Down Jun 19 '14 at 9:16
  • Could you explain what you mean by requisites? – user66369 Jun 19 '14 at 9:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy