0

I'm using busybox tools and I want to take all http links in a web page. I save an example link page by using curl or wget. However, it saves the page as html. How to do it with curl or wget commands?

example webpage = http://www.turanevdekorasyon.com/wp-includes/test/ 

The following data was saved in text format with firefox browser.

Index of /wp-includes/test/

Name <http://www.turanevdekorasyon.com/wp-includes/test/?ND>                                                                             Last modified <http://www.turanevdekorasyon.com/wp-includes/test/?MA>         Size <http://www.turanevdekorasyon.com/wp-includes/test/?SA>  Description  <http://www.turanevdekorasyon.com/wp-includes/test/?DA>

------------------------------------------------------------------------
up Parent Directory <http://www.turanevdekorasyon.com/wp-includes/>                                                                 28-May-2019 02:15        -       
[CMP] v1.0.zip <http://www.turanevdekorasyon.com/wp-includes/test/v1.0.zip>                                                                         28-May-2019 02:15       4k       
[CMP] v1.1.zip <http://www.turanevdekorasyon.com/wp-includes/test/v1.1.zip>                                                                         28-May-2019 02:15       4k       
[CMP] v1.2.zip <http://www.turanevdekorasyon.com/wp-includes/test/v1.2.zip>                                                                         28-May-2019 02:15       4k       

------------------------------------------------------------------------
Proudly Served by LiteSpeed Web Server at www.turanevdekorasyon.com Port 80
0

I'd suggest using the File | save As feature of Chromium and save the web page in MHT format after toggling the experimental "Save Page as MHTML" option on, by visiting the link "chrome://flags/#save-page-as-mhtml" in the Chrome browser.

  • I should be use curl or wget command in shell script file. – Ali Haydar Doğan Jun 5 at 23:02
  • If you want to use those specific tools, then you should ask for that in your question. Suggest you click edit and spell that out clearly. – K7AAY Jun 5 at 23:37
0

What's the point in using curl or wget? Use lynx:

lynx -dump 'www.example.com'

it will output all the links, shown and hidden.

  • I'm using busybox tools. Not every command is available. No lynx command. I need to recompile the operating system for lynx. I can't do that. – Ali Haydar Doğan Jun 6 at 10:32
  • Something like wget -qO- example.com | sed -n 's/.*href="([^"]*).*/\1/p' ? – Matteo Fabbroni Jun 6 at 10:44
  • Yes but this command doesn't give result. – Ali Haydar Doğan Jun 6 at 11:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.