0

I'm using a wget command which downloads an entire webpage with all assets, requisites and save it to custom folder for a later work on it.

My actual code:

$ wget --adjust-extension --span-hosts --convert-links --page-requisites \
 --no-directories --restrict-file-names=windows --no-parent \
 --user-agent="Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6"\
 ‐‐execute robots=off --limit-rate 400k --directory-prefix=elbache \
 ‐‐output-document=index.html http://www.bachecubano.com/celulares

By now everything works great, but I want to reduce the --page-requisites to only CSS and images (I don't want .JS files). The downloaded pages can be reduced by almost 40% if I don't load .js assets, which is not required for my purpose.

How do I config wget for only accept .css and /images but not *.js files?

  • there is an -A and -R option for accept/reject file types list. You can either accept the html, css, and images or reject javascript files. – ivanivan Jul 26 '18 at 19:13
  • examples please? I'm not expert. – Erich García Jul 26 '18 at 20:04
  • 1
    The man page makes it pretty clear - gnu.org/software/wget/manual/… – ivanivan Jul 26 '18 at 20:25
1

From the wget man page:

   -A acclist --accept acclist
   -R rejlist --reject rejlist
       Specify comma-separated lists of file name suffixes or patterns to 
       accept or reject. Note that if any of the wildcard characters, 
       *, ?, [ or ], appear in an element of acclist or rejlist, it will be
       treated as a pattern, rather than a suffix.

So you'd add the following switch to what you already have:

-A css,png,jpg,gif,jpeg

NOTE: You'll have to play with this and figure out what image formats this particular site uses.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.