2

I'm trying to write a script that notifies me when a static web page has changed. To do it, I'm using wget to download the web page, and diff to check whether it has changed or not. I'm running an Ubuntu 20.04 LTS virtual machine. Here is the example:

$ wget --quiet https://twiki.di.uniroma1.it/twiki/view/Reti_Avanzate/InternetOfThings2021 -O file1
$ wget --quiet https://twiki.di.uniroma1.it/twiki/view/Reti_Avanzate/InternetOfThings2021 -O file2
$ diff -q file1 file2
Files file1 and file2 differ

As you can see, diff reports differences between the two files. Why? Even if I try to compare them with diff -y they look the same for me.

UPDATE

Looking for differences with git diff --color-words -- file1 file2 gave the following result:

Differences

Apparently, there's a field in which the timestamp is added, and in one of the two files there's a <!--GENERATED_HEADERS--> which is absent in the other.

Any idea on how to solve it?

Improve this question
3
  • 2
    Have you tried a different viewer? git diff --color-words -- file1 file2 might be more illustrative than a line-based view.
    – ojdo
    Commented Jun 1, 2021 at 8:48
  • @ojdo thank you, updated
    – Francesco
    Commented Jun 1, 2021 at 9:03
  • 1
    wow, that's a weird piece of software. There's a bunch of links there that look like <a href="/twiki/..." class="twikiLink">, and the href and class attributes are in the opposite order in another between two downloads I got. You could use some tool to just extract the text content (but whitespace might still be a problem). Or, since it looks to be a wiki with version control, look at the revision number/date. It says "Topic revision: r170 - 2021-05-22" at the bottom of the page.
    – ilkkachu
    Commented Jun 1, 2021 at 9:21

2 Answers 2

Reset to default
5

You can solve this problem by using w3m with -dump option that ignores tags while rendering the page.

$ w3m -dump  https://twiki.di.uniroma1.it/twiki/view/Reti_Avanzate/InternetOfThings2021 > file1
$ w3m -dump  https://twiki.di.uniroma1.it/twiki/view/Reti_Avanzate/InternetOfThings2021 > file2
$ if cmp -s file1 file2; then echo "Files are not different"; fi
  Files are not different                                                              
$
Improve this answer
2

One more "answer" not on how to solve this, but what causes this: the used version of (apparently) TiddlyWiki seems to generate its output with random tag order in HTML elements. Examples:

--- file1
+++ file2
@@ -703,881 +703,881 @@
[...]
-<td valign="top" bgcolor="#ffffff" class="twikiTableCol2" align="left"> r1 </td>
[...]
+<td bgcolor="#ffffff" class="twikiTableCol2" align="left" valign="top"> r1 </td>
[...]

This repeats for attributes in tables, images, links.

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .