ext4 or NTFS lost all files of the last 2 years. How can I recover them?

Question

Problem

Suddently, all the files from feb-2015 to nov-2016 have disappeared from the filesystem.

Context

• Over the last 2 years, the system repeatedly terminated abruptly for power-fail reasons or for USB-hardware disconnections.
• This loss does not seem because of bad-blocks or things like this. It seems more something software-related.
• The disks are flat full-size vmdk files on a USB hardware-based RAID-5 connected to a VirtualBox Ubuntu.
• The host holding the RAID and the VirtualBox is a Windows.
• The RAID in windows is NTFS and the VirtualBox runs Ubuntu accessing the vmdk formatted in ext4. The Windows does not do other function than just containing a bunch of virtual machines and no user enters that Windows except to launch or stop virtual machines.

Situation

I've been using those disks (from the Ubuntu) on an intensive usage on a daily basis. And I have successfully used everyday the files from the previous day, and the previous week, and the previous month, and so on... Even after reboots... So the data must be there.

But suddently... my Dropbox started to delete files, I went to see what happened, and the fact is that... I can't tell if this happened "on the fly" or after a reboot.

But the thing is that... the latest file I can see is from jan-2015 while I've successfully been working with those disks until nov-2016 every single day. The files probably are in the blocks and for some reason I've lost all the index.

I can easily imagine that for some reason, the ext4 conserves backup-copies of i-node tables or something like this, and for some reason, the linux decided to "restore" the mapping to an old-one.

I've seen there are a bunch of tools. From running fsck to extundelete, and others.

What I see

Doing:

find . -newermt 20150201

gives me zero results.

In addition,

• For files I know they existed, I cd into the corresponding directories and I can't see the files there.
• I can see a directory structure I know I deleted.
• This has happened to several disks attached to the virtual machine.

ext4 or NTFS?

I have unmounted the RAID that contains the vmdk virtual disks from the Windows and I have mounted it read-only on a linux.

If I list the files, I can see this:

root@vagrant:/mnt/raid5/mnt# ls -la
total 1572864014
drwxrwxrwx 1 root root         4096 Jan 29  2015 .
drwxrwxrwx 1 root root         4096 Oct 31 21:14 ..
-rwxrwxrwx 2 root root 268435456000 Jan 26  2015 LinuxData250G_A-flat.vmdk
-rwxrwxrwx 2 root root          651 Jan 26  2015 LinuxData250G_A.vmdk
-rwxrwxrwx 2 root root 268435456000 Jan 26  2015 LinuxData250G_B-flat.vmdk
-rwxrwxrwx 2 root root          558 Jan 26  2015 LinuxData250G_B.vmdk
-rwxrwxrwx 2 root root 536870912000 Jan 26  2015 LinuxData500G_A-flat_3.vmdk
-rwxrwxrwx 2 root root 536870912000 Jan 26  2015 LinuxData500G_A-flat.vmdk
-rwxrwxrwx 2 root root          559 Jan 26  2015 LinuxData500G_A.vmdk

where the problematic disks are those LinuxDataXXXG_Y-flat.vmdk

What surprises me is that... the modification date of all those is Jan-26 of 2015... exactly "before" the time-loss in the content of the disks... I can't see any file from feb-2015 but I do see files from jan-2015.

I wonder if instead of being a problem with the EXT4 from inside the linux it could be a problem with the NTFS itself. What I can't tell is if the VirtualBox low-level drivers would update the time for the host files or that is "creation" time.

There is no space in the RAID to have a full-copy of the vmdks, so the lost data MUST be there somehow.

Working on an imaged-copy

I have already done an "image" of the original vmdk and I have unmounted the full RAID so I may be a bit "destructive" on the copy.

The thing is that it lasts 20 hours to copy the full backup image so I don't want to just be kidding with trial-and-error and I want to use the proper tools in the proper order.

Any failure in the process will be 1 day delay.

Questions

Assuming the data MUST be there... Assuming that from ext4 I can play around with i-node tables...

• Q1) What is the toolset I need to try to restore i-node mappings or re-create a new i-node table from nothing?
• Q2) Particularly, what is the order I have to run the tools? I mean, if I run fsck maybe I destroy things for extundelete or viceversa or for other tools.
• Q3) Is it true that i-node tables are backed-up from time to time into special partition areas reserved to that? Where? May I be lucky and find a backup of the tables if not from the "last day" at least from mid nov/2016 or so?

Maybe I am asking the wrong questions...

So one ring to rule them all: Q) How can I recover my lost files?

Answer 1

agreed USB might be okay to push backups over (still it is actually pretty pretty slow) but neither RAID over USB nor TCP-NETWORK over USB has been reliable and fast (yet).

1. backup faulty filesystem before messing with it (if the resources are there (additional external disks or NAS of sufficient size)
2. when backup is one, the user can safely run whatever tools
3. yes why not, start with fsck: fsck -y -v -f /dev/sdX
4. also highly recommended, set partition to auto-check on boot (before OS boots) tune2fs -C 2 -c 1 /dev/sdX1 (only works with ext3 and ext4)
5. apt install testdisk -> photorec /dev/sdx -> will bring back NTFS and ext4 files CONTENT but NOT /directory/structure/and/filename.jpg :(

good luck :)