5

I want to download a website from an URL, to view it locally, more exactly:

  • Download one single html page (no other linked html pages) and everything needed to display it (css, images, etc.)
  • Also download all directly linked files of type pdf and zip.
  • And correct all links to them, so the links do work locally.
  • The other links (for example to html files) should be kept untouched.

I'm open to all linux-based tools (macports support would be nice), using wget didn't work out for me so far.

Edit: wget -E -H -k -K -p is close to what I want, but how do I include pdf and zip files? (Source: Stackoverflow)

  • What did you try with wget? – cjm Oct 15 '13 at 21:17
  • I added my best attempt as an edit. – abenthy Oct 16 '13 at 8:35
3

try:

wget -r -k -l 1 --mirror yourSITE

Reason of -r along with --mirror:

-r: cuased wget acts recursively.

--mirror: cuased you tell wget behave such as mirror sites.Sometimes It needed.

-k : single documentation

-l 1 : level 1, one level recursively.

  • Tried that, but the level restriction isn't working. It basically mirrors the whole site. – abenthy Oct 16 '13 at 8:34
  • 1
    From the wget documentation: ‘--mirror’ Turn on options suitable for mirroring. This option turns on recursion and time-stamping, sets infinite recursion depth and keeps ftp directory listings. It is currently equivalent to ‘-r -N -l inf --no-remove-listing’. It seems -l 1 --mirror overrides -l 1 with -l inf. See my answer omitting the --mirror option. – tamarintech Oct 16 '13 at 13:49
  • 1
    better readeable explanation: explainshell.com/explain?cmd=wget+-r+-k+-l+1+--mirror+yourSITE – rubo77 Nov 7 '14 at 3:20
2

HTTrack (homepage) can mirror sites for off-line viewing with a rather fine grained options as to what to download and what not. It is also able to resume interrupted downloads.

1

Did you try wget -rk -l 1 <sitename> or similar? That should recursively fetch the content while converting the links to relative paths. The end result may be more .html files than you want, but you can remove all of them except for index.html which you should be able to browse normally.

  • -m,--mirror is also an interesting option. – peterph Oct 15 '13 at 22:00
  • 1
    mirror is not suitable. OP suggests he only wants the first level. Mirroring the site sets unrestricted recursion. – tamarintech Oct 16 '13 at 13:47
  • Looks good, but linked css files generated by php scripts (for example css-style.php?option1=bla) don't work. I couldn't figure out how to get them working yet. – abenthy Oct 17 '13 at 13:50
  • Right, that is to be somewhat expected. There are some dynamic pieces of content that wget may not fetch or that may be fetched in a static manner. Also, I think some static content like <link> specified content may not be fetched. I haven't used the other product suggested (HTTRack). Does it do a better job? – tamarintech Oct 17 '13 at 14:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.