20

I am trying to using the following dd command to clone remote xfs disk to another location.

ssh root@source_ip  "dd if=/dev/vda2 bs=16M conv=noerror,sync status=progress " | sudo dd of=/dev/nvme1n1 bs=16M conv=noerror,sync status=progress

However, I got the following error msg

828895133696 bytes (829 GB) copied, 39610.432258 s, 20.9 MB/s
1288490188800 bytes (1.3 TB) copied, 64529.849170 s, 20.0 MB/s
dd: error writing '/dev/nvme1n1': No space left on device

The source disk has 1TB and the target disk has 1.2TB.

Can anyone help to explain why can't perform the disk to disk clone in this situation? Thanks!

I want to try to recover the deleted files from the source disk, and I'm not sure whether DD is the only right tool for this situation.

Improve this question
12
  • 1
    Please remove conv=sync from the right-hand dd and try the command line again. If that still gives you errors, please add iflag=fullblock to the right-hand dd.
    – Mark Plotnick
    Jun 10, 2020 at 4:07
  • 9
    dd is not a tool for cloning and will clone every single sector into the new disk even if it's unused. Use a proper cloning tool instead. They're smarter and much faster
    – phuclv
    Jun 10, 2020 at 10:39
  • 3
    You're using Linux here based on your tags, I very strongly recommend that when cloning disks over the network, you don't use a network pipe, but use a rpoer network block storage protocol to expose the target disk on the source system or vice versa. NBD is what I tend to prefer for this, but ATAoE and iSCSI work too. This will let you use a proper cloning tool like ddrescue and turn most network reliability issues into performance issues instead of letting them corrupt data.
    – Austin Hemmelgarn
    Jun 10, 2020 at 13:40
  • 2
    use xfsdump and xfsrestore since you are dealing with an XFS file system. Many online articles telling on how to do specifically what you are trying to do.
    – ron
    Jun 10, 2020 at 15:04
  • 3
    Incidentally, your title says you're cloning a disk. Your command shows you cloning a partition of a disk. Are you sure the target is (at least) the same size as the source?
    – Chris Davies
    Jun 11, 2020 at 8:39

4 Answers 4

Reset to default
31

Compare this answer (note bs=1K is used there):

dd is an old, cranky program initially intended to operate on tape devices. When you tell it to read one block of 1kB, it attempts to read one block. If the read returns less than 1024 bytes, tough, that's all you get.

conv=sync gets important when dd indeed reads less than it expected. In your case any time the pipe from ssh fails to deliver a full 16M block (because of bs=16M) to the local dd in one read, the conv=sync of the latter will fill the "missing" data with zeros (NUL bytes). But the real data is not missing. What the local dd considers missing will be delivered when it tries to read the next block.

In effect the local dd injects zeros in random(-ish) places of the stream. In case it's not obvious I emphasize: this corrupts the data, the resulting "clone" will be virtually useless.

Using conv=noerror,sync even on the remote (reading) side may be wrong. Compare this answer: What does dd conv=sync,noerror do? You need to really know how it works to use it properly.

For the local dd conv=noerror,sync you used makes little sense. Drop it or add iflag=fullblock (if supported). Frankly I'm not sure if conv=noerror,sync iflag=fullblock has any advantage over not using these options at all in a case when dd reads from a pipe.

Improve this answer
9
  • 14
    Better still, don't use dd at all
    – Chris Davies
    Jun 10, 2020 at 13:22
  • 3
    @roaima: what should be used, then?
    – a3nm
    Jun 10, 2020 at 20:05
  • 9
    @a3nm cat should do, or pv for a nice progressbar. pv /dev/sda > /dev/sdb
    – gronostaj
    Jun 10, 2020 at 20:31
  • 13
    Yes, in this example OK. But for more complicated use cases, dd has options (seek, skip, etc.) that you can't replicate with just cat
    – a3nm
    Jun 10, 2020 at 20:42
  • 3
    @a3nm in those situations consider using dd. But generally for straightforward replication of disks or disk partitions, dd is most definitely not the way to go. (And that's what this question is about.)
    – Chris Davies
    Jun 10, 2020 at 22:50
23

As others have already identified, the problem is that you're using dd. Rule #1 of using dd is you do not use dd. Rule #2 of using dd is you do not use dd. Rule #3 of using dd (for experts only) is don't use the bs option.

There is no magic in dd. The magic is in /dev/*.

dd will happily mangle your data and setting a block size may or may not help performance. Only use dd if you've figured out how to use it safely and you've done benchmarks in your specific use case to determine block sizes that are faster that letting the system do its thing.

ssh root@source_ip  "cat /dev/vda2" | sudo tee /dev/nvme1n1 >/dev/null
Improve this answer
6
  • 2
    Probably worth ssh root@source_ip "gzip </dev/vda2" | zcat | sudo tee /dev/nvme1n1 >/dev/null for all those potentially empty blocks
    – Chris Davies
    Jun 10, 2020 at 13:02
  • 5
    @roaima Yes, good point. Or simpler: ssh -C.
    – Gilles 'SO- stop being evil'
    Jun 10, 2020 at 13:04
  • What advantage does tee provide here?
    – Binarus
    Aug 24, 2023 at 8:20
  • 1
    @Binarus sudo tee myfile is one way to write myfile as root. sudo sh -c 'cat >myfile' is another. sudo cat >myfile wouldn't work since the surrounding shell would open myfile, and it's not running as root: only cat runs as root, but cat doesn't open its output, it just writes to its standard output.
    – Gilles 'SO- stop being evil'
    Aug 24, 2023 at 9:30
  • Thank you very much, and +1. I have got your point. I didn't come to that idea because I'm mostly working as root in such cases (not normally).
    – Binarus
    Aug 28, 2023 at 8:27
8

For anything involving forensics, including undeleting files, dd is the RIGHT tool (at least for the read).

The problem, as others have pointed out, is that at the end of a ssh pipe, you aren't reading it blocked. So you either need iflag=fullblock on the second dd, or to just skip the second dd altogether.

flag=noerror is probably wrong. You want to know if it failed.

flag=sync is often a mistake. It tells dd to override the OS's buffering. You don't need that. At most, run sync afterwards. And for reading, it is meaningless.

Two different status=progress entries is a mistake. Both will report overwriting each other. You really just want to know how it finished. Omit status=...

bs=16M seems nice, but ideally bs should match the hardware block size. For classic disks, that was 512 bytes. More modern ones use 4k. I would recommend 4k, 16k, or 64k.

For disks that aren't full, compression is also good. The ssh -C solution is probably easiest.

So you could write:

ssh -C root@source_ip  "dd if=/dev/vda2 bs=64k" | sudo dd iflag=fullblock of=/dev/nvme1n1 bs=64k

or

sudo sh -c 'ssh -C root@source_ip  "dd if=/dev/vda2 bs=64k" >/dev/nvme1n1'

(this one might have ssh issues)
or

sudo chown `id -u` /dev/nvme1n1
ssh -C root@source_ip  "dd if=/dev/vda2 bs=64k" >/dev/nvme1n1
sudo chown root /dev/nvme1n1

or use the rather clever solution involving using tee.

testing

If you want to use the double dd form, it is probably good to test that you've got the right options before doing the full transfer. Add count=100 to the first dd so that it does a little bit of the disk. You should see both status lines in a few seconds and they should show the same amount transferred. If it works, repeat with the count=100 removed to do the full disk.

Improve this answer
3

The command dd was modeled after the command DDR (Disk Dump and Restore) from IBM mainframes from the 1960s and is mainly intended to convert and reblock block oriented I/O from/to block oriented disks. This is however not what you intend to do here.

The important problem in your use case is than you have an unreliable network pipe in the middle that does not grant to read specific block sizes.

If you are using a real dd program (not sure what you use since your output is incompatible to the standard) you should get output like this:

"%u+%u records in\n", <number of whole input blocks>,
    <number of partial input blocks>

"%u+%u records out\n", <number of whole output blocks>,
    <number of partial output blocks>

You need to use dd in a way that makes sure that you only get numbers > 0 before the plus sign and a 0 after the plus sign.

Since there is no grant to get a specific amount of data when reading from the ssh output pipe, you need to make sure to always try to read less than you would get. You could still use the following command to read from the source disk:

dd if=/dev/xxx bs=16x1024k

without problems, but you need to use this command to read from the pipe and to write to the destination disk:

dd ibs=1b obs=16x1024k of=/dev/yyy

Do not user other options (in special none of the non-standard option you are using). If the dd command that writes to the destination disk finally writes:

nnnn+0 records in
mmmm+0 records out

you have been successful and dd did never read less than 512 bytes from the input and thus could correctly reblock the output.

If you see partial blocks (> 0 to the right of the plus sign) for the input blocks, you need to repeat the copy process.

Note that because of the large output blocksize, you of course may get one partial block written to the destination dist at the end. So the summary message for the dd that writes to the destination disk may alternatively be:

nnnn+0 records in
mmmm+1 records out
Improve this answer
3
  • 1
    If you have a block size that doesnt cleanly divide to the data size you can get a +1 at the end due to the last block being incomplete.
    – masterX244
    Jun 10, 2020 at 12:04
  • OK, but only for the output blocks. I'll correct that.
    – schily
    Jun 10, 2020 at 12:15
  • I suspect dd is based on the DD statement in IBM mainframe's JES2 or JCL environment (early to mid 1960s). That actually specified the mapping between an internal filename and an external filename, and could also do certain actions on the file, like deleting it, or releasing a tape. It was not unusual for the work of a job to be in the DD statements, and the program to be IEFBR14 (a standard return immediately program). The DD statement's syntax tended to be: //<internal_name> DD <keyword>=<value>,...
    – David G.
    Jun 11, 2020 at 1:40

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .