Automatically save the text-only of every web page visited?

Question

I'd like to automatically save the text of every web page I visit, to it's own corresponding text document, not html document. I am not looking to save all the styling and additional repeated html tables etc of code and other un-needed wasted bytes. I would love to be able to save the valuable information only, just the text information from every web page visited.

So I imagine the files of web-page text could be saved looking something like this.

Website_Title.txt

and contain only the corresponding web page's text only inside of the document. Kinda exactly like my xclip script below but save the entire page automatically. Gathering filename title from the corresponding web-page's <title></title> tags.

Can this be done? Possibly with the help of the app 'xclip'? I imagine probably not on it's own. Since 'xclip' doesn't do things automatically, I don't think...

---

Currently I am using 'xclip' to save text that I select after pressing a keyboard shortcut.

Like so, for example... if you wanted to save this selected text below:

In my case, I would press: Ctrl + Shift + X

would then save a file that looks like this:

and inside of that file would look like this:

---

---

I'd like to be able to do the same thing, but instead of selecting and saving the text I want, it would save the entire page, (like if you were to use Ctrl + A) and then save the content, and append a filename with the website's title, automatically, every page.

Thank you very much in advance for your valuable time.

---

---

Oh and if anyone wants to use my xclip-save-selection.sh script, here it is:

#!/bin/sh
#
#           _  _                                               _           _    _                    _    
# __ __ __ | |(_) _ __  ___  ___ __ _ __ __ ___  ___  ___ ___ | | ___  __ | |_ (_) ___  _ _      ___| |_  
# \ \ // _|| || || '_ \|___|(_-</ _` |\ V // -_)|___|(_-</ -_)| |/ -_)/ _||  _|| |/ _ \| ' \  _ (_-<| ' \ 
# /_\_\\__||_||_|| .__/     /__/\__,_| \_/ \___|     /__/\___||_|\___|\__| \__||_|\___/|_||_|(_)/__/|_||_|
#                |_|                                                                                      
#
# Save Selected Text Script
# XFCE4: Applications > Settings > Keyboard
# Attach this script to a custom keyboard shortcut to be able to save selected text from anywhere

xclip -o > "/home/anonymous/.logs/clips/$(date +'%Y-%m-%d_%H-%M-%S')_$(xclip -o | cat -s | perl -pe 's/\r?\n/_/' | perl -pe 's/\ /_/g' | sed 's/    /_/g' | sed 's/__/_/g' | sed -e 's/^M//' | tr -s -c [:alnum:][:blank:] _ | cut -c1-50).txt"
bash -c 'notify-send "Save Selected Text - Success!"'

Answer 1

I would do it like this:

• Install w3m
• w3m -dump url-of-site > output.txt

You could then also create a script to look at your history, scrape the urls and do the text conversion from them. Also, make it so it looks at the text file names for a reference of sites already converted and ignores processing them, so as to not create duplicates.