9

My girlfriend has a external hard disk with 10 years+ of photos, documents and more. A lot of these files originate from her old iPhone 5 and her MacBook. The hard disk itself is NTFS Format. Since the disk is so old, it turns into a hazard of data loss (what an irony).

As we tried to upload all the files into OneDrive to store them safely, we got 1,000s of errors because of invalid file names. I realized that many files started with ._, e.g. ./pic/92 win new/iphone/._IMG_1604.JPG. I don't understand macOS and why files should be named like that, but for sure you can never get them into OneDrive like that.

So I decided to hook it to my Raspberry Pi and rename all files with the wrong characters from the command line. After listing the nearly 10,000 files, I ran the following over the whole hard disk.

find . -name "._*" | sed -e "p;s/\._//" | xargs -d '\n' -n2 mv

Furthermore, I removed some leading whitespace in filenames with zmv.

I tried the command in a test environment first and it looked fine. But I didn't check the file size.

After my girlfriend connected the hard disk back onto her Mac, all renamed files show a file size of 4KB (empty)! I screwed it up and I don't know how.

I assume the data is still there, but I somehow screwed the filesystem.

Does anybody understand what I did wrong? More importantly, do you see a chance to recover the files? I would appreciate any advice.

Improve this question
6
  • 4
    First problem was not using mv -i which would stop rather than clobber existing files.
    – stark
    Commented Mar 6, 2021 at 17:35
  • 3
    Did you already upload the files to onedrive? I mean, ignoring the errors about the ._ files (which are irrelevant anyway, see my answer), were the rest uploaded? If so, everything is absolutely fine, all files have been backed up and only the ._ files were ignored which is not a problem at all.
    – terdon ♦
    Commented Mar 6, 2021 at 17:42
  • 2
    @PeterCordes if it was an HFS variant the resource fork would have held the metadata. Here it's the ._* files holding the metadata
    – Chris Davies
    Commented Mar 7, 2021 at 12:30
  • 5
    Most important thing is to leave the damned disk alone. Do not write to that disk at all. Do not even allow it to be attached to a booting OS just in case. When you are ready to do data recovery (see answers below) mount it manually READ ONLY.
    – Zorawar
    Commented Mar 7, 2021 at 14:55
  • 3
    And if you want to be ultra safe, don't even mount it at all and make a disk image with dd or some similar tool since there might be some exceptional situations whereby mounting itself can write data, but I'm not sure how this goes with NTFS.
    – Zorawar
    Commented Mar 7, 2021 at 15:09

7 Answers 7

Reset to default
20

As mentioned by terdon, when writing to a "foreign" filesystem, Mac OS uses two filenames for each file. One with the actual contents and a second one with metadata that would have been stored in the resource fork. You renamed the metadata filename to the content filename, thus deleting the content file in the process.

However, I slightly disagree with his that the originals were overwritten. The data should be in the disk (I hope it's not a ssd), but you no longer have a filename to them, and the clusters will be marked as free space.

If the files were uploaded into OneDrive, you already have a copy there. An advantage here is that you have the full list of filenames that were originally in the disk. If you don't, continue reading.

First of all, before doing any further recovery on the disk, you should make a copy and work with that, e.g. with dd. This way, you avoid making things worse during a recovery attempt, since you would be working on a copy of the data.

Second step would be to attempt recovery with a tool like ntfsundelete, trying to recover the deleted entries.

Third, since these files were presumably copied in full from a different system, I expect the files wouldn't (generally) have been fragmented, but using sequential blocks, so it will probably be possible to recover most of them through file carving.

In that case, a tool like photorec should be able to find most of the photos, even with no access to the filesystem metadata.

Finally, remember to back up what you might recover!

Good luck

Improve this answer
2
  • 2
    I said that the files were overwritten, not that the actual blocks on the hard drive were overwritten. That's why I sad that "You might be able to get some back using data retrieval approaches but it will not be easy." Your answer describes precisely the kind of data retrieval approaches I was thinking of.
    – terdon ♦
    Commented Mar 7, 2021 at 14:36
  • Upvoted for the advice to make a volume backup first.
    – Peter - Reinstate Monica
    Commented Mar 9, 2021 at 16:43
9

Now it's hard to recommend anything, but if I were you, I'd try using R-Studio Undelete - in my experience it's the best application for restoring accidentally deleted or damaged data from a number of filesystems including ext2, ext3, ext4, fat32, NTFS and others. Just stop making any changes to the partition with files, please (no more renames, copying, anything). Anything you're writing to it at this point makes restoring data harder if not impossible.

A slightly irrelevant piece:

When you notice an unexpected inconsistency in your filesystem data the first thing to do is to stop writing to your device or this filesystem, then you take your device to a professional/service center.

If a professional is out of reach or too expensive, then you create a full partition image, and then you work only with this image.

You never continue to work with the source filesystem or even device because the inconsistency could be caused by a hardware failure and further writes (and sometimes even reads) may exacerbate the issue.

Improve this answer
0
9

If you haven't overwritten the files, you can trivially recover all of your files. If they are images, your best bet is to use a JPEG file carving tool. JPEGs are easy to carve because they are somewhat self-validating. Photorec from CGSecurity in an open source tool that I recommend.

If you want more fun, you can use the OpenSource program Autopsy

Before you do anything else, you should make a forensic disk image of the drive. You can do that with ewfacquire, which is part of libewf.

Improve this answer
9

I'm afraid you have overwritten all of the original files. The ._ files are a special thing of macOS systems and HFS drives. From what I understand, they seem to be used to store things like the icon associated with a file. You can see this post on Ask Different for more details.

Basically, for every foo file you create, there will be a ._foo hidden file created by the system automatically. So when you renamed these to remove the ._, you renamed them to match the name of the actual, real file with your data, effectively deleting the real file and keeping only this "meta-file".

So no, the files are not still there, you have overwritten all of the originals. You might be able to get some back using data retrieval approaches but it will not be easy. Since you have deleted the files, you will be forced to try recovery tools. There is no simple way to get them back.

For future reference, the command you ran is very destructive. The two major flaws are:

  1. You did no checking for file name collisions and would overwrite existing files. So if any of these files shared a name with an existing file (in your case, this was true for all of these files), then the existing file would be overwritten. Next time, use mv -i which will prompt before overwriting.
  2. You assumed no file names contain newlines. Yes, that is likely to be a safe assumption, but you don't know and if any file existed with a newline in its name, your mv command would fail and possibly mv the wrong thing.
Improve this answer
15
  • 3
    The directory entries for the files are overwritten, but the file data itself is not overwritten.
    – vy32
    Commented Mar 7, 2021 at 13:52
  • 4
    "So no, the data is not still there, you have overwritten all of the originals". This is likely not correct. The data will hopefully still mostly be there to be recovered. Do not give the impression that all is lost, because it is not by the sounds of it. mv does not overwrite data like dd does.
    – Zorawar
    Commented Mar 7, 2021 at 14:38
  • 1
    @Zorawar since the wording seems to be confusing many people, I clarified that I am only talking about files and not the underlying data. My point is that the files have been deleted as effectively as if the OP had run rm file. So yes, they should be retrievable using classic data retrieval tools, but there's no simple solution, the files haven't simply been moved somewhere else. The OP will need to look at data retrieval tools.
    – terdon ♦
    Commented Mar 7, 2021 at 14:45
  • 4
    Well, the poster may well not know these things. I would personally not write "overwritten all of the original files" without further clarification because for anybody else in this situation this will be misleading for them. They will likely not know about inodes/IDs and filesystem internals etc.
    – Zorawar
    Commented Mar 7, 2021 at 15:02
  • 3
    @terdon, your terminology is confusing. When you say that the files are overwritten, you appear to mean that the directory entries are overwritten. The file contents remain. Typically people are more concerned with recovering the file contents than the file metadata such as the file name or the modification date. Your answer gives the OP that all hope is lost, and this is not the case.
    – vy32
    Commented Mar 7, 2021 at 16:06
4

Sorry that you lost your data. As other have pointed out, it may be hard to recover the files because they are now marked as free data in your hard drive. First thing, only mount that system read-only, to avoid overwriting the data.

As you are looking for JPEGs, take a look at recoverjpeg, which is optimized to look for images in raw discs -- see there: http://www.rfc1149.net/devel/recoverjpeg. There are Debian packages for it.

Improve this answer
0

I have had success using Runtime Software's GetDataBack for NTFS but it will only run on Windows (your question is asking for Linux recovery software, I think).

I wanted to include it as an alternative in case you could not recover what you needed. I've used this software for years and it is excellent.

Improve this answer
0

A while ago I have had excellent general results with the Windows tool recuva. What it can do in your case depends on how mv is implemented and maybe how blocks are allocated. Perhaps all recovered jpegs will have 4k garbage data on top, which could easily be cropped out, given today's image sizes.

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .