2

Having a HDD of 1TB (931.5G). I did:

sudo sh -c 'cat /dev/sdf >disk_1t.img'

But it stopped with read error at 908G.

Thought next step, (should have been first), is to use gnu ddrescue. Problem is that I do not have a map file. Questions:

A) Is there any risk of damaging the existing image using --generate-mode?

ddrescue -n --generate-mode /dev/sdf disk_1t.img disk_1t.map

Guess -n option is best for first run. Speak out if false.

If I understand it correctly, it would then start to read at 908G?

I do not want it to start reading whole disk once more.

B) Is there any point in trying to get it to read at say 907.5G in case last bytes are corrupted? Would cat write bad data in any case? Would it perhaps be better, if possible, to start reading at say 910G, skip 2G as they likely is at a part causing trouble? Then do 908-910 on a later run.

C) Is there a way to do a check of the first 908G after the remaining 23.1G has been read / attempted / exhausted?


If any point, here is tail of the existing image:

$ tail -c 512 disk_1tb.img | xxd
00000000: b481 e803 8733 0000 5801 b454 6e3a 9e51  .....3..X..Tn:.Q
00000010: 4cc8 4851 0000 0000 e803 0100 2000 0000  L.HQ........ ...
00000020: 0000 0000 0000 0000 9314 260b 9414 260b  ..........&...&.
00000030: 9514 260b 9614 260b 0000 0000 0000 0000  ..&...&.........
00000040: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000050: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000060: 0000 0000 817a 168b 0000 0000 0000 0000  .....z..........
00000070: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000080: 1c00 0000 0000 0000 0000 0000 c8e9 3caa  ..............<.
00000090: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000000a0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000000b0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000000c0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000000d0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000000e0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000000f0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000100: b481 e803 b650 0000 7300 b454 6e3a 9e51  .....P..s..Tn:.Q
00000110: 879d c950 0000 0000 e803 0100 3000 0000  ...P........0...
00000120: 0000 0000 0000 0000 9714 260b 9915 260b  ..........&...&.
00000130: 9a15 260b 9b15 260b 9c15 260b 9d15 260b  ..&...&...&...&.
00000140: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000150: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000160: 0000 0000 827a 168b 0000 0000 0000 0000  .....z..........
00000170: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000180: 1c00 0000 0000 0000 0000 0000 f0cc 5056  ..............PV
00000190: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000001a0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000001b0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000001c0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000001d0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000001e0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000001f0: 0000 0000 0000 0000 0000 0000 0000 0000  ................

1 Answer 1

1

In your case, there is no real need to generate a mapfile. Since you know the offset exactly, you can pass it along as input position (if in doubt, make the input position a little smaller. Making it larger will leave a gap. Note that ddrescue considers G as GB, not GiB so you have to specify units correctly.)

ddrescue --min-read-rate=1MiB \
         --input-position=900GiB \
         /dev/sdx sdx.img sdx.map

The map file will result in something like:

#      pos        size  status
0x00000000  0xE100000000  ?

Which marks the 0-900GiB region as ? to be considered later. As long as you keep using the --input-position parameter around, ddrescue will simply ignore that area and only deal with the rest.

This is what you can do if you are sure that the previous part was already fully copied. Which is your situation unless you did something fancy that might have ignored errors or changed offsets and thus led to data corruption (like when using dd conv=noerror,sync).

If you change your mind and want to re-read the first 900GiB anyway, you can simply remove the --input-position parameter.


Is there any risk of damaging the existing image using --generate-mode?

In general it takes effort to make ddrescue do the wrong thing. ddrescue only damages the image file if you make it use the wrong source device (damage by copying the wrong data) or make it use the wrong offset --output-position (damage by copying the right data to the wrong place). Then there's also the even more exotic --fill-mode that writes data patterns to the image file, but you'd have to go out of your way to actually use that.

If I understand it correctly, it would then start to read at 908G?

In generate mode, ddrescue does not know how the image file came about (by ddrescue, by dd, by something else), so it makes the assumption that any binary zero area might have been skipped over previously. So if you have areas with all-zero data on your disk, it might attempt to re-read those, and only skip over areas that have non-zero data.

In theory, not re-reading all sections that contain data would save you some time, but generating the map after all requires reading all data in the image file first, so effectively it'll be only faster than starting over if your source drive is slow (due to damage).

The main point of the --generate-mode is not so much to save time (it takes time to generate) but to avoid overworking an already failing source drive, and to facilitate resume when it's unclear which data segments had been copied previously.

Is there any point in trying to get it to read at say 907.5G in case last bytes are corrupted?

In your case not at all, technically you can resume where you left off and not re-read a single previously copied sector. There should not be corruption in your existing image unless you ignored the error (like dd conv=noerror,sync does), but there is no such risk with cat.

Is there a way to do a check of the first 908G after the remaining 23.1G has been read / attempted / exhausted?

You can use cmp to compare two files or block devices:

cmp /dev/sdx sdx.img

This can also be done at an offset:

cmp --bytes=64M /dev/sdx sdx.img 900G 900G

However, the data will only match if the drive was only ever accessed read-only. If you have anything that automounts, it will have modified filesystem metadata.

So in theory, cmp should not find a difference for the first 908G but in reality, it often just happens anway as many distros come with various automatisms these days that are difficult to disable.

4
  • Great. I use a USB bay docking and have not mounted it, though it asks what to do I decline any action (but one never know). And, yes, the time used is not an issue - it is that I do not want to stress the disk more then needed (as you point out as well). I thought of cmp as well - but isn't it better to use ddrescue to compare at the end in case it find any bad data or false positives due to meta changes? For example --size=908G.
    – Turmund
    Sep 7, 2021 at 12:15
  • Impossible to say perhaps. If data has changed due to corruption on source after first run doing ddrescue could perhaps duplicate that corruption to the target.
    – Turmund
    Sep 7, 2021 at 12:17
  • I notice this a bit too late, also only by actually looking at the map file with 0xD18C2E2800 offset, that ddrescue actually defaults to SI (power of 1000) units so if you give it 900G it actually starts at ~ 845GiB or so. So for ddrescue you actually have to write MiB, GiB. (900GiB should be like 0xE100000000 not 0xD18C2E2800...). So I eat my words, turns out ddrescue actually screws you over like this, I didn't know... ^^; Sep 7, 2021 at 12:47
  • @Turmund you'd have to create a 2nd copy with ddrescue and then compare the copies, or at least I'm not aware of ddrescue having a data comparison feature. It's up to you how to process the data it rescued. Sep 7, 2021 at 12:51

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .