If I try wget on a webpage, I am getting the page as html. Is it possible to retrieve only text of a file without associated html ? (This is required for me since some of the HTML pages contains c program is getting downloaded with html tags. I have to open it in browser and manually copy the text to make a .c file.)
wget will only retrieve the document. If the document is in HTML, what you want is the result of parsing the document.
You could, for example, use
lynx -dump -nolist, if you have lynx around.
lynx is a lightweight, simple web browser, which has the
-dump feature, used to output the result of the parsing process.
-nolist avoids the list of links at the end, which will appear if the page has any hyperlinks.
As mentioned by @Thor,
elinks can be used for this too, as it also has a
-dump option (and has
-no-references to omit the list of links). It may be especially useful if you walk across some site using -sigh- frames (MTFBWY).
Also, keep in mind that, unless the page is really just C code with HTML tags, you will need to check the result, just to make sure there's nothing more than C code there.
If you don't have these other tools installed, only wget, and the page has no formatting just plain text and links, e.g. source code or a list of files, you can strip the HTML using sed like this:
wget -qO- http://address/of/page/you/want/to/view/ | sed -e 's/<[^>]*>//g'
This uses wget to dump the source of the page to STDOUT and sed to strip any < > pairs and anything between them.
You can then redirect the output of the sed command to the file you want to create using > :
wget -qO- http://.../ | sed -e 's/<[^>]*>//g' > downloaded_file.txt
NB: You may find that it has extra whitespace in the file that you don't want (e.g. lines are indented a few columns)
It may be easiest to use your text editor to tidy the file up that (or a source formatter as you're downloading C source code).
If you need to do the same simple thing to every line of the file you could include a command to do that in the sed command (here stripping one leading space):
wget -qO- http://.../ | sed -e 's/<[^>]*>//g;s/^ //g' > downloaded_stripped_file.txt
just to add another tool. I prefer
w3m, which is a
lynx like console browser. You may want to check out whats already available on your system.
w3m -dump website.html