I am trying to create a script to monitor a web page and send a telegram notification whenever there is a change in that page, I am using diff to accomplish this task.
The script seems to work well but some web pages insert a kind of random ID in the content of the page, this ID changes every time I download the page and I need to solve this for the diff to work correctly.
I need to find some way to remove/edit this randomly generated ID, in short I need to edit the string of this ID deleting almost all letters, spaces, hyphens, numbers, etc. and save only the data without ID.
Example, I need to modify only the information that is enclosed in the quotation marks " ":
<path d="M0 0h7v7h-7zM9 0h1v2h-1zM12 0h1v4h-2v-1h-1v-1h1v-1h1zM16 0h1v3h-1v-1h-1v-1h1zM18 0h4v1h-1v1h1v1h-2v-2h-1v1h-1zM23 0h1v1h-1zM26 0h7v7h-7zM1 1v5h5v-5zM22 1h1v1h-1zM27 1v5h5v-5zM2 2h3v3h-3zM8 2h1v1h1v1h1v1h1v1h-1v1h-1v-1h-1v1h1v1h-2zM14 2h1v1h-1zM23 2h1v2h1v3h-1v-2h-4v-1h3zM28 2h3v3h-3zM15 3h1v1h2v2h-1v-1h-2v2h-1v-1h-1v-2h2zM18 3h1v1h-1zM19 5h1v1h-1zM12 6h1v2h-2v-1h1zM16 6h1v2h1v-2h1v1h1v-1h1v1h1v1h-2v1h1v1h1v1h1v1h-3v1h-1v-1h-2v1h-1v-2h-2v1h-1v-4h2v1h-1v1h2zM22 6h1v1h-1zM23 7h1v1h-1zM0 8h1v1h1v-1h5v1h-3v1h3v1h-1v1h-1v-1h-2v-1h-1v1h-1v1h-1zM22 8h1v1h-1zM24 8h1v1h-1zM26 8h5v2h1v2h-2v1h3v1h-1v1h-1v1h-1v-2h-1v-1h-1v-3h1v1h1v-1h-1v-1h-1v1h-2zM9 9h1v1h-1zM23 9h1v1h-1zM32 9h1v1h-1zM8 10h1v1h-1zM18 10v1h2v-1zM10 11h1v1h-1zM25 11h1v1h-1zM3 12h2v1h-1v2h-2v-1h1zM6 12h3v1h-1v1h1v1h-1v1h-1v-1h-1v1h1v1h-1v1h-2v1h-1v-1h-3v-5h2v1h-1v2h1v1h1v-1h2v-2h2v-1h-1zM11 12h1v2h3v-1h1v1h1v1h-1v2h-1v-2h-1v1h-3zM14 12h1v1h-1zM17 13h2v1h-2zM22 13h6v1h-1v2h-1v1h-1v-1h-1v-2h-2zM20 14h2v1h1v1h-2v-1h-1zM9 15h1v1h-1zM28 15h1v2h-1v1h1v1h1v-1h-1v-1h2v1h1v1h1v3h-1v-1h-1v-1h-1v3h-1v-2h-1v-1h-2v-1h1v-1h-1v-1h1v-1h1zM10 16h1v1h-1zM17 16h1v1h-1zM32 16h1v2h-1zM8 17h2v1h-1v1h-1v1h2v3h-1v-1h-1v1h-2v1h2v1h-3v-1h-1v1h-1v-1h-1v-2h1v1h2v-1h2v-1h-1v-1h1v-1h-1v-1h2zM11 17h3v2h1v-1h1v1h1v1h-1v1h1v1h-2v-2h-3v-1h1v-1h-1v1h-1v1h-1v-2h1zM16 17h1v1h-1zM19 17h1v1h-1zM21 17h1v1h-1zM23 17h1v1h-1zM18 18h1v1h-1zM20 18h1v1h1v1h-1v1h-1v1h-1v-1h-1v-1h2zM22 18h1v1h-1zM24 18h2v1h-1v1h-1zM1 19h2v1h2v1h-3v-1h-1zM5 19h1v1h-1zM11 20h1v1h1v1h-1v1h-1zM23 20h1v1h4v2h-2v1h4v1h-1v2h1v-2h1v1h1v1h-1v1h-1v3h1v1h-1v1h-1v-1h-1v-1h1v-1h-1v-1h-4v-1h-1v-2h1v-4h-1v1h-1v-2h1zM0 21h2v1h-1v3h-1zM31 22h1v1h1v1h-3v-1h1zM10 23h1v1h-1zM13 23h1v1h-1zM16 23h1v1h-1zM21 23h1v1h-1zM9 24h1v1h1v-1h2v2h-1v1h-1v1h-1v-1h-1v-1h-1v-1h1zM14 24h1v2h-1zM17 24h1v3h2v-1h-1v-2h1v1h2v1h-1v1h1v1h-1v1h-1v1h-1v1h-3v2h-2v-1h1v-1h-4v-2h5v1h2v-1h1v-1h-2v1h-1v-2h-1v-1h1v-1h1zM22 24h1v1h-1zM25 25v3h3v-3zM32 25h1v1h-1zM0 26h7v7h-7zM26 26h1v1h-1zM1 27v5h5v-5zM8 27h1v1h1v3h1v2h-1v-1h-1v-1h-1zM12 27h1v1h-1zM2 28h3v3h-3zM31 28h2v2h-2zM21 29h2v1h-2zM20 30h1v1h-1zM23 30h1v2h-1v1h-1v-1h-1v-1h2zM26 30h2v1h-2zM8 32h1v1h-1zM17 32h3v1h-3zM24 32h1v1h-1zM26 32h2v1h-2zM31 32h1v1h-1z"/>
Result I need:
<path d = ""/>
or anything like these examples:
<path d="0"/>
<path d="CLEAN"/>
<path d=""/>
<path d=/>
I believe that it may be possible to resolve this issue using sed, but due to the complexity of the string, there are many characters, spaces, hyphens, numbers, etc, I am having a lot of difficulties to get to the ideal command
Example of the script I am using:
#! /bin/bash
page_mofication="$(cat /opt/pagename/listing/latest_modifications/latest_modifications.log)"
fileold=/opt/pagename/latest_modifications/latest_modifications_old
filenew=/opt/pagename/latest_modifications/latest_modifications_new
log=/opt/pagename/listing/latest_modifications/latest_modifications.log
logold=/opt/pagename/oldfiles/latest_modifications/latest_modifications.log
mv $log $logold-`date +%d-%m-%Y_%H:%M:%S`
wget https://www.pagename.com -O $filenew
diff $fileold $filenew >> $log
message=$'\n'"$page_mofication"
/etc/scripts/telegram-send.sh "$message"
cp $filenew $fileold
exit 0
Any ideas on how I can solve this problem?
https://shellcheck.net
, a syntax checker, or installshellcheck
locally. Make usingshellcheck
part of your development process.