[ EDIT:
Answered my own question.
Used unison and some hacky post-processing
(
copy-pasted the log output of unison,
tweaked it in my text-editor with multi-selection editing,
then did some shell script processing on that (fishshell)
)
((I've got so much more disaster recovery stuff to slog through, so I guess I'm done with this for now...))
]
So I have an SSD that used to be inside another computer,
and I put it in one of those little SATA-to-USB enclosure/adapter shell things,
mounted it as an external data drive,
and used rsync -aAX to copy the boot partition over into a dir on this computer, for a backup.
But then, after some other events that probably didn't change the contents of the original boot partition,
I made a second backup.
So now I have two dirs on this computer,
which I think are probably two copies of exactly the same backup,
but I want to make sure.
So my question is:
What is the best way to compare/diff these two big backup directories?
Summary of things I have considered/tried but had problems with or am unsure about:
diffitselfrsync"dry run" trickunison[(just thought of, but it hasn't finished running yet, due to the large size of the backups and the slow speed of my old hardware.)]
Is one of these essentially a good option?
If so, any corrections to the details of how I should go about using it?
Or are there any separate, additional options I should know about..?
Details of my attempts and the results/problems:
diff
The obvious way of doing it for "normal" directories would be like:
$ diff -r dir_A dir_B
(
or maybe $ diff -r --no-dereference dir_A dir_B?
I don't know; I honestly don't properly understand the function of --no-dereference
-- it's just something I've found got me the results I wanted in vaguely similar situations in the past.
)
However, the problem with using diff is that these dirs are of course very unusually large,
and full of "weird" files from the bootable system
(eg "character special files" and "block special file" etc).
rsync
So it occurred to me to just use rsync again between them,
doing a "dry run" and seeing if it reported any changes it would make.
Like:
$ sudo rsyncy -n -aAX dir_A dir_B --log-file=log_file
However, it then occurred to me
- "what if there were new files in dir_B?"
- "would rsyncy necessarily always report that?"
So I guessed you would have to check both:
$ sudo rsyncy -n -aAX --delete dir_A dir_B --log-file='log_file[A-to-B]'
and
$ sudo rsyncy -n -aAX --delete dir_B dir_A --log-file='log_file[B-to-A]'
which is starting to feel a little suspiciously like maybe that isn't really the right tool for the job after all...?
The log files I got read:
A-to-B
#=>
2023/07/21 01:43:04 [26686] building file list
2023/07/21 02:12:24 [26686] sent 80.58M bytes received 292.46K bytes 45.93K bytes/sec
2023/07/21 02:12:24 [26686] total size is 229.29G speedup is 2,835.29 (DRY RUN)
B-to-A
2023/07/21 01:41:58 [26406] building file list
2023/07/21 02:12:15 [26406] sent 80.58M bytes received 292.50K bytes 44.49K bytes/sec
2023/07/21 02:12:15 [26406] total size is 229.29G speedup is 2,835.29 (DRY RUN)
which are (ignoring the timestamps and speed information),
annoyingly ALMOST the exact same:
both
sent 80.58M bytes
but tiny different received:
received 292.46K bytes
vs
received 292.50K bytes
So yeah, again, I'm feeling doubtful this rsync trick is really the right tool for the job...?
Maybe the correct answer really is like:
"
Just be patient and let diff run for ages to process the two huge directories.
(You can just ignore all the error messages about special file etc.)
"
??
unison
[not sure yet?]
diffonly on them?findcommand and send its output todiffsomehow? (Like with the-execflag or something?) Or...?diffonly on them.