2

Is there a way to check if a web page changes using the command line? Something like this:

$ check-for-changes https://bbc.co.uk/web-page

https://bbc.co.uk/web-page just changed

Using a headless browser or some other mechanism?

2
  • This is not clear-cut. "Web page" usually means a lot of different files (contents, icons, images, ...). You have to have a clear idea what changes you are interested in.
    – vonbrand
    Feb 12, 2021 at 15:33
  • I am interested in changes to the html. this is for a parcel tracking site - I'd like to see when there is any changes/progress to the delivery without loading every 5 mins.
    – beekay
    Feb 12, 2021 at 15:40

2 Answers 2

4

The site may serve pages with a LAST-MODIFIED header: do a HEAD request and extract that header.

curl --silent --head https://glennj.github.io | grep -i '^last-modified:'

Not all sites return that header though.

2
  • the site has a Date response header, but not last-modified. The page is dynamically generated, so the 'Date` is always the time of request. I guess I would need to know when the html changes.
    – beekay
    Feb 12, 2021 at 15:40
  • Have you experimented with DopeGhoti's answer? Feb 12, 2021 at 17:53
3

First, get a fingerprint for the current state as a baseline:

curl --silent www.example.com | md5sum > www.example.com.md5

Then, you can subsequently check to see if the fingerprint changes:

curl --silent www.example.com | md5sum > www.example.com.md5new
if ! cmp www.example.com.md5 www.example.com.md5new > /dev/null; then
    printf "%s has changed from baseline!\n" "www.example.com"
fi
rm www.example.com.md5new
1
  • ! cmp -s www.example.com.md5 www.example.com.md5new if you don't need the actual diff.
    – Kusalananda
    Feb 13, 2021 at 14:22

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .