Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I use wget's built-in spider mode as a convenience sometimes to quickly check a local site for broken links. This morning I turned its attention to a production site that we'd just put major changes on, and it's coming up with 3 broken links, but it seems impossible to tell where they are! (It only says what they're linking to and there's no straightforward way of relating that alone back to a page.)

The options I'm currently using are wget -r -nv --spider http://www.domain.com/ -o /path/to/log.txt. Does anyone know of an option I'm overlooking, a way to read the output, or even a simple substitute for this command that will also let me know what file the links appear in (and ideally a line #)?

share|improve this question
add comment

1 Answer 1

You should be able to watch the web server logs, in conjunction with the wget run. Look for the 404's in the log file and pull the referrer field. That will tell you the page that contains the broken link.

It should then just be a matter of examining that page for the offending link.

share|improve this answer
    
Good idea. I forgot I asked this on here, actually! What I ended up doing was using it in combination with grep on my local copy of the site (particularly using the -n option to get line numbers). –  user19866 Jul 10 '12 at 19:52
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.