I have a 1.5 TB partition of data, of which I accidentally overwrote some bytes at the beginning, due to a typo like:

ssh somewhere command | dd of=/dev/sda3     // should have used quotes here, dd was executed locally by mistake!

/dev/sda3 holds a LUKS container for the 1.5 TB ext4 partition with important data.

When I noticed the problem I suspended and killed the dd; it should have overwritten less than 4K.

Is there a way to recover the data? The computer has not been rebooted since, so whatever data was lost may still be in RAM? What do the first (say) 4k of a LUKS container contain?

The partition is still mounted, but shows errors like

[1157706.786897] EXT4-fs error (device dm-4): htree_dirblock_to_tree:896: inode #2: block 9249: comm ls: bad entry in directory: rec_len % 4 != 0 - offset=0(0), inode=2791782547, rec_len=44529, name_len=90

when trying to access it.

Please help!


PS: I made some more tests and it seems more data was overwritten than just 4K :-( But still a very very small percentage of the 1.5 TB data! Can I still dump data from the uncontaminated region - maybe search with an ext4 recovery tool (if a good one exists) in a dump of the /dev/mapper/cr_sda3 - would that still work?

  • 1
    Do you have a backup of the header? See the warnings section of man cryptsetup... – jasonwryan May 12 '15 at 22:34
  • Sorry, no backup. Never happened to me before :-( -- but the container is still unlocked, and the unlocked key should be in RAM?! -- that is, if it is the key which is lost?! – Ned64 May 12 '15 at 22:37
  • 1
    Try and generate a new passphrase for your next slot: if that succeeds, you know you aren't completely hosed (which would be miraculous). – jasonwryan May 12 '15 at 22:44
  • Please explain a little more or give a pointer, thanks! I am no expert on LUKS. – Ned64 May 12 '15 at 22:50
  • BTW is the volume not still unlocked? – Ned64 May 12 '15 at 22:51

First: run dmsetup table --showkeys. Save the output of that somewhere safe—that big long hex string it shows is the actual encryption key (master key) used to secure your data. LUKS works by (simplification here) encrypting that key with your passphrase(s), so keep in mind that compromise of that key means game over—a passphrase change won't help. You have to wipe and recreate the LUKS partition. However, that same property means that even with a completely destroyed LUKS header, you can use that "table" (including the key) to read your data.

The line you're looking for (and there may be many lines, LVM also uses Device Mapper) looks like this. Except instead of a bunch of 0's, you'll have random hex digits (the 0s is what you get without --showkey):

Zia-swap_crypt: 0 11714560 crypt aes-xts-plain64 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 0 253:1 0

(You do the above first because it's quick. If the power goes out, the machine crashes, etc. you can use it to recover your data. Without it, your data would be unrecoverable.)

You want to keep the entire line. Better yet, the entire output. You can feed that line back to dmsetup to restore the table, and thus your access to the data, even without the LUKS header).

Next, copy an image of the decrypted device somewhere. The decrypted device is the name you see in the dmsetup output above; in my case /dev/mapper/Zia-swap_crypt. It's the same one you'd put in /etc/fstab or pass to mount.

Now, you can copy your data off the running system (e.g., with tar) or, if that fails, attempt an fsck to repair the filesystem. (Then copy data off).

You can use that key to create a new LUKS header, and that should work—or just reinitialize it and start over.

In the future, please make use of cryptsetup luksHeaderBackup.

  • Thanks, I have made a copy of the key now. I am also dumping the /dev/mapper/cr_sda3 to a file (that may take a few days as I have to buy a new hard disk to finish the dump (1.5 TB partition!!!)). Is it correct that the first few MB of the data are not recoverable? If so, how do I recover data from the ext4 partition of which the first few MB are broken? (I remember something about backup superblocks...) – Ned64 May 12 '15 at 23:45
  • @Ned64 if it's still mounted, you might just be able to use tar. Other than that, I suspect fsck will be able to get it mountable. But you want to do that on a copy (maybe even a copy of a copy, or at least set up a snapshot—fsck may also destroy things). You should be able to use one of the backup superblocks. You may have lost more than the first however many bytes you overwrote; e.g., by destroying filesystem metadata about other data—but that metadata is spread out, so you probably only hit a little of it. – derobert May 12 '15 at 23:47
  • 1
    @Ned64 BTW: You want the whole line. Maybe the output of dmsetup info -c as well. Those will make your life easier should you have to use them (the other things can be brute forced fairly trivially, but not having to do so is much easier. The encryption key, of course, can not be brute forced, at least not with the computing power available on Earth.) – derobert May 12 '15 at 23:49
  • @Ned64 considering those errors, tar probably won't get everything. But at each step you want to get as much as you can. So first get what you can with tar, then fsck one of your copies, and get what you can from that. Compare, see if there is anything critical missing. If anything critical is missing, you can figure out your next step. Some types of files (e.g., JPEG images) can be recovered using media scan undelete tools. – derobert May 12 '15 at 23:53
  • 1
    @Ned64 You really shouldn't be trying e2fsck until after your dd is done. But probably best to ask a new question with the e2fsck errors, to see if anyone knows how to get the FS repaired. I take it then that tar wasn't able to get anything off? :-( – derobert May 13 '15 at 17:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.