2

Intro

I've accidentally done the dreaded thing. I types ls instead of cd, and then a few commands later ran rm -rf on my current directory instead of a copy of this directory from a few days ago which appeared in mnt after a script I wrote didn't do what I expected.

I'm aware that there are already questions like this on here, but none are comprehensive. I'd like to take the "opportunity" to start a public contrib series of answers which can be used as a reference for others in future.

For example, there are suggestions about using photorec and testdisk. These might not be the best solutions. I found another answer, here, which suggested using locate. I don't know anything about this or how it works.

My System

My system is a Debian 10 system tracking the Testing branch. The data was on a SSD, formatted as ext4. As soon as I realized that I had run rm -rf in the wrong place I held the power button down on the computer to turn it off.

Steps to Solution

I have another laptop running another Debian system. I have taken the SSD out of the computer and plugged it into an external USB to Sata interface.

I believe I now need to mount this in a way which means the system cannot write data to the device. How should I do this?

Can I then use this locate method (linked question) to get the data back? Before running any of this I want to make sure that it is the right thing to do.

Is the data important?

Yes - the directory I deleted contains all the code for my PhD. I have a couple of weeks left. I have an older version which I can use, and then re-implement everything which I did during the last few week or so. However in that short period of time I changed a lot of stuff. I made a "breakthrough" with the research this week. The way in which the code is written means it is not at all trivial to re-implement this.

Did I have backups? (Edit)

The only one I know for sure that I have is on a server with the date 2 May 2020. I've got a huge amount of work done since then as I've been pretty much working around the clock.

There might be other backups on this SSD, but some of them are in my home directory - so those will be rm -rf'd, and the backup script was not working as I expected - so probably the regular backups from that do not exist. (Trying to delete the weird stuff that was being copied is how I ended up in this mess.)

At the moment I can't access the SSD to check the date. I need to know how to mount it such that data cannot be written to it, if that is possible.

If I only have the ones from 2 May then that's really very bad. I've been working crazy hours on this recently, which is probably why I woke up this morning and accidentally nuked it.

6
  • Boot from a live system, then mount your external backup drive. From the live system you can also backup your current state by using dd on the entire partition (to an external drive) before experimenting with restore tools. You can also use that dd image as an external image to a virtual machine from where to try to restore, without ever touching your real partition. – Ned64 May 11 '20 at 13:15
  • The locate answer you linked to is only about finding the names of files that are missing, it won't help you get the data back. Mounting read-only is easy (sudo mount -o ro /path/to/your/disk /path/to/mountPount, but restoring deleted data is not easy at all. – terdon May 11 '20 at 13:20
  • @terdon No, sorry! Mounting with -o ro does write to disk! Don't do it! You can only copy block by block, e.g. by dd, or loop-mount ro but not like that. – Ned64 May 11 '20 at 14:21
  • @Ned64 what do you mean? The -o ro should mount it read-only. What would be written to disk? – terdon May 11 '20 at 14:24
  • 4
    @terdon Metadata is written - which might overwrite blocks previously allocated for files which have now been deleted. Here is a reference: linux.die.net/man/8/mount from there (option -r, --read-only): "Note that, depending on the filesystem type, state and kernel behavior, the system may still write to the device. For example, Ext3 or ext4 will replay its journal if the filesystem is dirty. To prevent this kind of write access, you may want to mount ext3 or ext4 filesystem with "ro,noload" mount options or set the block device to read-only mode, see command blockdev(8)." – Ned64 May 11 '20 at 14:30
3

Here are a few things to consider:

  • Do a full disk copy first: dd if=/dev/sdX of=image.img bs=1M status=progress
    Work on this copy (e.g. photorec image.img) instead of the disk itself.

  • In my (long) experience, foremost is better than photorec for many filetypes. Depending on your filetypes you might want to look into adding custom file headers to foremost, this has been VERY effective for me in the past. More on foremost/photorec in my answer here

  • This answer of mine might help you find usable files out of the recovered haystack.

  • Do NOT mount the SSD. As already stated in the comments, mounting readonly might not always be enough to prevent overwrites. Work on dd images.

Now here's a (mindblowing) trick I learned a while back that might actually work for you.

the directory I deleted contains all the code for my PhD

So it's text. Now if most of your code is in one (or few) file(s), you can actually try this:

grep -ai '<this text is in my newest code revision>' image.img

I know it almost sounds too easy, but it saved me before in the same situation. You might want to add -C x to the command to include x (replace with a number) lines above and below the matched line.

Note that your file(s) might be partly corrupted. Try many patterns. Start with many words to try and find an exact match, if there's no results, try less words.


And the obligatory... ALWAYS make backups. NOT on the same drive. VERIFY backups. You're dealing with text, I assume this isn't hundreds of gigabytes so you might aswell set up cloud backups (encrypted of course) or something like that.


Another word of advice: Don't spend more time freaking out over this than you would need to re-write the code. Stay calm, try what you can, see if my grep method helps you and if you come up with nothing after a whole day the best solution is probably to get your latest backup and work again from there.

1

The start of an answer

I used photorec from the testdisk utility suite to recover some data.

List of things I tried

  • Boot Debian 10 Laptop (seperate machine)
  • Insert SSD via USB to SATA controller (BAD IDEA!)
  • Disk automatically mounted (don't know how to prevent this)
  • Started copying data using dd to make backup
  • Ran out of space
  • Cancelled dd job
  • Might be a good idea to dd copy data first, but don't know how to do it
  • Problems I have: disk is 1TB in size, and I need to copy all of that 1TB (probably? can I ignore zero'd blocks?) before examining with photorec
  • Internal SSD (laptop) is only 500 GB
  • Attached external 4TB drive
  • unmounted 1TB SSD (one I'm trying to recover from)
  • Didn't bother with the dd
  • Loaded photorec
  • instructions here: https://www.cgsecurity.org/wiki/PhotoRec_Step_By_Step
  • only thing I did differently was change the file types it searches for, turning them all off and searching only for .tx? and .txt related options (which includes .c source files etc)

Processing of the data

  • Photorec produced hundreds of output folders and thousands of files
  • I don't understand the naming/numbering convention for file names or directory names
  • File names were all jumbled nonsense
  • Used grep -rIw to search for exact match whole word strings in subfolders
  • Did some processing on the output using a bash script to remove irrelevant lines
  • used diff to compare resulting matches
  • Found the latest version of one of my source files
  • Several more to go...

Summary

Not sure whether this should be the "accepted answer" - but the above method is what I used to retrieve my files.

It's not a particularly good method but works.

For the case where the data lost is images, it is probably a lot easier to flick through things or view thumbnails to find the right files.

Text files are harder to deal with because the system creates many false positives due to text files the system generates, logs etc.

0

If you were fast enough with the power-off then, after a full binary backup of your partition, you could give extundelete a try.
If I remember correctly, extundelete utilizes the journal on your partition to 'undo' the modifications of the directory files caused by your rm -rf.

2
  • I did try that. But from another system. It said it couldn't recover any files, but I don't know if I did what I was supposed to. How quickly would I have had to power off the system for this method to work? – user3728501 May 12 '20 at 2:58
  • That would depend on how much write activity occurred after the command rm rf. – bey0nd May 12 '20 at 13:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.