3

Is there any to get or find the referrers of a link that were crawled by wget?

Suppose that some link returns a 400 error. wget only shows the url on which the 400 error occurrs, not the referrer of error url. If possible please provide any utility similar to wget in linux or wget option for doing this.

1 Answer 1

2

The referrer is passed as part of the request header. Wget prints it if you turn on debug with -d. Then use grep to find the "referer" (note that debug output is probably written to stderr).

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .