Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

Is there any to get or find the referrers of a link that were crawled by wget?

Suppose that some link returns a 400 error. wget only shows the url on which the 400 error occurrs, not the referrer of error url. If possible please provide any utility similar to wget in linux or wget option for doing this.

share|improve this question
add comment

1 Answer 1

The referrer is passed as part of the request header. Wget prints it if you turn on debug with -d. Then use grep to find the "referer" (note that debug output is probably written to stderr).

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.