Sign up ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. It's 100% free, no registration required.

I am trying to download all the links from There are 7 of them, excluding the domain which I wan't to ignore. Yes, so that means, I don't want to download links that start with domain. Also, I would want them saved in a .txt file, line by line. So there would be 7 lines. Here's what I've tried so far. This just downloads everything. Don't want that.

wget -r -l 1
share|improve this question
Yes, wget downloads whole pages. Why do you think it might only save the links? – michas Feb 26 '14 at 6:44
Well wget has a command that downloads png files from my site. It means, somehow, there must be a command to get all the URLS from my site. I just gave you an example of what I am trying to do currently. – Ali Gajani Feb 26 '14 at 6:46
You're trying to use completely the wrong tool for the job, this is not at all what wget is designed to do. In future, please don't assume a particular tool for a job when asking for a question; it's not a good way to ask questions in general, but it's especially poor practice in a technical venue. – Chris Down Feb 26 '14 at 6:52
Apologies, I am a noob to wget. I thought wget is powerful functionality built in for tasks like web crawling and more, so I was assuming it would do something like this. I did see the man page for wget and didn't find anything w.r.t what I wanted, so I came here. – Ali Gajani Feb 26 '14 at 7:08

2 Answers 2

up vote 8 down vote accepted

wget does not offer such an option. Please read its man page.

You could use lynx for this:

lynx -dump -listonly | grep -v > file.txt

From its man page:

          for -dump, show only the list of links.
share|improve this answer

As others have pointed out, wget is not designed for this. You can however parse its output to get what you want:

$ wget -O - 2>/dev/null | 
    grep -oP 'href="\Khttp:.+?"' | sed 's/"//' | grep -v facebook > file.txt

That creates a file called file.txt with the following contents:
share|improve this answer
Please don't parse html with regex: – michas Feb 26 '14 at 21:43

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.