3

I was trying to resize my LUKS crypt following this https://wiki.archlinux.org/index.php/Resizing_LVM-on-LUKS and I got to the partition resize with parted and seriously screwed things up. I typed 870 as the new size and forgot to put a G on the end. It shrunk my partition down to 870M I immediately resized it to 870G but by then the damage was done. Luckily I could still decrypt the LUKS crypt but I couldn't get my Logical Volume to even have a device file on the system. LVM recognized the volume as existing and showed the device file it was attached to but the file didn't exist and it showed it as having no filesystem. I did vgscan --mknodes and it successfully generated the device file but testdisk still wouldn't show it. I recreated the volume and put a new ext4 filesystem on it and now testdisk will show the drive but scanning yields nothing. I get a whole bunch of ext4 entries but all of them either say Can't open filesystem or No files found. Is there anyway for me to recover the filesystem that was on the disk? I don't want to write any data to it until I get what's on it off of it unless that's not possible.

EDIT: After poking around the real thing I need help with is recovering files from a previous ext4 filesystem. My drive had an ext4 system on it and that has since been overwritten with a new one however all the data from the old system still exists as shown by sudo dd if=/dev/Storage/Storage bs=1M | strings -fn 16. The only thing I did after my screw up was put a new ext4 FS on and nothing else so most of my data is probably still intact. I need to recover that data.

pvdisplay shows the following

--- Physical volume ---
PV Name               /dev/mapper/Storage
VG Name               Storage
PV Size               931.51 GiB / not usable 3.68 MiB
Allocatable           yes (but full)
PE Size               4.00 MiB
Total PE              238466
Free PE               0
Allocated PE          238466
PV UUID               CAueGx-Glzx-zCd0-H00m-R8d5-KTRc-9Ff7ay

--- Physical volume ---
PV Name               /dev/mapper/sda3_crypt
VG Name               mint-vg
PV Size               118.50 GiB / not usable 0   
Allocatable           yes 
PE Size               4.00 MiB
Total PE              30336
Free PE               10
Allocated PE          30326
PV UUID               UJJfu8-S2Ac-pEZl-PlPa-uUzJ-axEs-ckbDWG

My backup shows

# Generated by LVM2 version 2.02.98(2) (2012-10-15): Thu Aug 13 20:45:52 2015

contents = "Text Format Volume Group"
version = 1

description = "Created *before* executing '/sbin/lvreduce --config log{command_names=0} -f -l 217600 /dev/Storage/Storage'"

creation_host = "desktop"   # Linux desktop 3.19.0-25-generic #26~14.04.1-Ubuntu SMP Fri Jul 24 21:16:20 UTC 2015 x86_64
creation_time = 1439523952  # Thu Aug 13 20:45:52 2015

Storage {
     id = "lM3S9T-inH1-mKsq-5doN-H8hT-zO3F-LF9jDx"
    seqno = 2
    format = "lvm2" # informational
    status = ["RESIZEABLE", "READ", "WRITE"]
    flags = []
    extent_size = 8192      # 4 Megabytes
    max_lv = 256
    max_pv = 256
    metadata_copies = 0

    physical_volumes {

        pv0 {
            id = "nH1Axo-5nBo-WcyA-Xc4E-KwRt-K0Ib-ScK8Ch"
            device = "/dev/mapper/Storage"  # Hint only

            status = ["ALLOCATABLE"]
            flags = []
            dev_size = 1953520999   # 931.511 Gigabytes
            pe_start = 2048
            pe_count = 238466   # 931.508 Gigabytes
        }
    }

    logical_volumes {

        Storage {
            id = "Qb01kz-y1RG-PVQp-cGjB-sj77-xgnJ-w9kn3n"
            status = ["READ", "WRITE", "VISIBLE"]
            flags = []
            creation_host = "desktop"
            creation_time = 1436247513  # 2015-07-06 22:38:33 -0700
            segment_count = 1

            segment1 {
                start_extent = 0
                extent_count = 238466   # 931.508 Gigabytes

                type = "striped"
                stripe_count = 1    # linear

                stripes = [
                    "pv0", 0
                ]
            }
        }
    }
}
  • Your metadata kind of blows my theory, no segment fragmentation and no other LV in there so changing it back to original size should've worked out OK. – frostschutz Aug 14 '15 at 21:29
  • @frostschutz well right now I'm using extundelete to pull any residual backup data I had from previous backups off of other drives but the last time I actually made a backup was months ago so my drives are coming up empty. The data's probably already been overwritten on them. As far as that drive is concerned I can't seem to restore my partition. testdisk won't find it. I'm halfway inclined to make a backup of my disk and just go back to using it until I can figure it out on the backup but I don't really have a drive big enough to handle a backup. – Scoopta Aug 14 '15 at 21:32
2

You can't fix LVM by growing size back to original size, unless you were very lucky and the LV had no fragmentation whatsoever due to previous resizes. Chances are the new LV will have the first 20G or so of your original filesystem but the remaining 780G (or whatever) are scrambled eggs (wrong data, wrong offset, wrong order).

And that's assuming you're using HDD media. If it was SSD, with issue_discards=1 in your lvm.conf, the data would simply be gone, which is why I never use this option.

You have to check /etc/lvm/{archive,backup}/ for old versions of your metadata. Each file in there says when it was created, for example:

description = "Created *before* executing 'lvremove HDD/mdtest1'"

You're looking for the one that says Created before lvresize 850 with the G missing. And then vgcfgrestore LVM metadata using that backup and hopefully then it will be back in working order.

If you do not have such files in /etc/lvm, either because you did this from a Live CD that lost this data, or the damage happened on your root LV, things get a bit more complicated as you have to hope for the LVM metadata on disk to contain this bit of history in its circular buffer.

Rough method to see what's possibly in there:

dd if=/dev/pvdevice bs=1M count=1 | strings -w -n 16
  • I haven't done a config restore but everything is there. Every file and everything shows using dd like that. Idk if there's any corruption but basically all of the files seem to show still. Now to retrieve it. – Scoopta Aug 14 '15 at 9:53
  • @Scoopta: I updated the first paragraph of my answer in order to explain the problem in more detail. You should not mount this, particularly not in read-write mode, especially not fsck, until you've verified the new LV is using the same extents as the old one did. – frostschutz Aug 14 '15 at 10:01
  • It was on an HDD not an SSD and I haven't mounted it. I don't want to poke the badger. The only thing I've done with it is run it through testdisk which didn't yield much. The results from dd look good but I'm still not sure how to recover the data. Should I just restore the config? – Scoopta Aug 14 '15 at 10:03
  • 1
    I do have /etc/lvm/archive. I was saying the dd | strings result looks promising. I'll give the vgcfgrestore a go and hopefully with any luck I'll have files again. – Scoopta Aug 14 '15 at 10:20
  • 1
    I updated my post to have the info you asked for. – Scoopta Aug 14 '15 at 20:04
1

If you hadn't blown away the old ext4, there might have been some hope for fsck to do some repairs and find some intact directory structures.

There might actually still be hope for that, by using an alternate superblock that was in the part of the disk you didn't mess up with mkfs. Or if your old FS had a different number of backup superblocks than your current empty FS, there might be some that weren't overwritten.

e2fsck -b some-offset

This applies only if your LV is mapped to the same bytes as the old one (which Frostschultz talked about in his answer).


You can recover a few types of files that can be picked out of a binary stream. (Since you have no metadata mapping filenames to byte ranges.) Some files have a recognizable header, and it's possible to tell where the end-of-file is. As long as your files are fragmented, there's some hope of recovering some nameless files.

Foremost is a tool for doing this. It claims to finds jpg, gif, png, bmp, avi, exe, mpg, wav, riff, wmv, mov, pdf, ole, doc, zip, rar, htm, and cpp files.

  • I recreated the FS because udev wouldn't give my LV a device file without doing so. I had one for the drive and the PV but the LV which is where all my data was had no dev file so I figured I'd have to do it. Maybe that wasn't a good idea but I couldn't get testdisk to scan the volume because it had no dev file. Anyway I'll give foremost a try and see if I can't find some superblock somewhere. testdisk found a bunch but couldn't read any for whatever reason. – Scoopta Aug 15 '15 at 5:12
  • @Scoopta: obviously avoid writing to the new filesystem on that disk until you've run foremost, since your data is there in what the FS sees as free space. – Peter Cordes Aug 15 '15 at 5:26
  • 1
    Yep that's exactly what I'm planning on doing. I'm actually imaging the disk with dd so I can at least go back to using it and work on recovering the backup. But if I can recover it sooner rather than later then I won't need the backup – Scoopta Aug 15 '15 at 5:35
  • So foremost is working fantastically for the most part. I'm able to get most of the data I want I think. Right now I'm pulling jpegs and it's working pretty good and it supports java files which is what I want to get off the drive the most although I haven't actually tried to pull them yet since my jpegs are still coming off. That being said recovering a superblock would be preferable because I have game saves that are obviously not supported by foremost and I'm not sure what if any recognizable header they have for manual addition. Do you have any tips on determining the offsets I should use? – Scoopta Aug 15 '15 at 8:04
  • 1
    Yea...well the java files aren't built in they're provided in the default config and disabled by default which I didn't realize until after telling it to do my jpegs but I already understand what you mean. I have jepgs from every game I had installed, the android sdk, things I deleted months ago and god knows what else. But I'll probably run it a second time with java files enabled in the cfg and hopefully get most of my stuff back. – Scoopta Aug 15 '15 at 8:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.