2

This evening, I had to hard shut down my computer after some kind of kernel panic.

When I rebooted, I noticed my ~/.ssh/id_rsa had been replaced with an empty file.

Rebooting to a USB and running fsck on my home partition reported that the filesystem was in good shape.

This alone is not a problem. I access to the original key. However, I am concerned that other files may have been similarly truncated.

My last backup, using deja-dup, was three days ago, so I could just do a full roll-back, but I would rather just ask deja-dup what files have changed since then and look for "suspicious" files.

This seems to be exactly the purpose of duplicity verify, so after some man page skimming, I tried:

duplicity verify --verbosity 4 --no-encryption file:///path/to/backup/ /home/${USER}

which ran to completion without reporting changes. At a minimum, I expected my ~/.ssh/id_rsa to be detected, but I have added, removed, and changed other files.

My next try was then the same, but with the --compare-data flag:

duplicity verify --verbosity 4 --no-encryption file:///path/to/backup/ /home/${USER}

Which seems to report that every file in my home folder is new, starting like:

Local and Remote metadata are synchronized, no sync needed.
Last full backup date: Fri Dec 15 11:43:22 2017
Difference found: File . has permissions 1000:1001 700, expected 0:0 555
Difference found: New file .AndroidStudio2.3
Difference found: New file .AndroidStudio2.3/config
Difference found: New file .AndroidStudio2.3/config/inspection
Difference found: New file .AndroidStudio2.3/config/inspection/Default.xml

I have had Android Studio installed for months, so it was most certainly in my backup from three days ago, and ls reports that Default.xml still exists and is 108 bytes long.

As a final effort, I changed the target directory to /, since that seemed to be the root when using duplicity list-current-files, which required adding some regular expressions to limit duplicity to only consider my home folder:

duplicity verify --verbosity 4 --compare-data --no-encryption --include-regexp ".*home/${USER}/\.ssh.*" --exclude-regexp ".*" file:///path/to/backup/ /

Which had the interesting effect of reporting that my home folder doesn't exist:

Local and Remote metadata are synchronized, no sync needed.
Last full backup date: Fri Dec 15 11:43:22 2017
Difference found: File home is missing
Difference found: File home/${USER} is missing
Difference found: File home/${USER}/.AndroidStudio2.3 is missing
Difference found: File home/${USER}/.AndroidStudio2.3/config is missing
Difference found: File home/${USER}/.AndroidStudio2.3/config/inspection is missing
Difference found: File home/${USER}/.AndroidStudio2.3/config/inspection/Default.xml is missing

At this point, I am certainly just misunderstanding how I should use duplicity. How can I verify a backup generated by deja-dup?

duplicity list-current-files has output starting:

Local and Remote metadata are synchronized, no sync needed.
Last full backup date: Fri Dec 15 11:43:22 2017
Tue Feb  6 19:36:56 2018 .
Wed Aug  2 17:32:09 2017 home
Tue Feb  6 00:38:20 2018 home/${USER}
Sat May 13 18:49:24 2017 home/${USER}/.AndroidStudio2.3
Thu Jun 22 19:42:14 2017 home/${USER}/.AndroidStudio2.3/config
Sat May 13 18:57:45 2017 home/${USER}/.AndroidStudio2.3/config/inspection
Sat May 13 18:57:45 2017 home/${USER}/.AndroidStudio2.3/config/inspection/Default.xml
  • To be clear, the various outputs where I have listed ${USER} actually had my user account name, I have just replaced them for privacy – Sompom Feb 16 '18 at 15:27
  • Sompom, can you try duplicity verify --compare-data --no-encryption --file-to-restore home file:///path/to/backup/ / ? – ede Feb 17 '18 at 19:11
  • @ede Thanks for the reply! I have actually been running that command for an hour, and it appears to be working. If you would like to post an answer I will accept it (once it runs to completion and I verify it does what I need), otherwise I will self-answer – Sompom Feb 17 '18 at 19:52
  • @ede with --file-to-restore and --compare-data duplicity seems to have done something useful, in that it correctly noticed lots of files had been modified/added/deleted since the backup was taken, but it still has not detected ~/.ssh/id_rsa is the wrong size. At this point, I am not sure that --compare-data does what the man page seems to say it does! – Sompom Feb 17 '18 at 20:45
2

@ede and I found the same solution at the same time, in my case on the duplicity mailing list

duplicity verify needs the --compare-data flag in order to verify the on-disk files, and it needs the --file-to-restore flag in order to look in the proper directory, so the final command that solved my problem is:

duplicity verify --verbosity 4 --compare-data --file-to-restore=/home/${USER} --no-encryption file:///path/to/backup/ /home/${USER}/

Unfortunately, this still does not detect that ~/.ssh/id_rsa was damaged. At the same time, trying to restore from the backup restored a 0-byte file... It's quite possible something happened to my file weeks ago.

  • hey Sompom, you correctly found out that duplicity has a quirk where in/exclude only works while looking at the local storage. for restore/verify you will need --file-to-restore=path to limit the incrementing over the backup contents. --compare-data actually is needed if you want to actually coompare against the local data, w/o it the backup will just be tested for successful restorability. ..ede/duply.net – ede Feb 18 '18 at 11:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.