14

I want to back up my SSD using the Linux dd command, but I'm not sure how reliable that method will be. I think I read somewhere that dd does not check for or report errors, so obviously if true then it will be a deal breaker.

This will be the command:

sudo dd status=progress bs=512K if=/dev/nvme0n1 of=/media/d/ssd.img

So please explain how reliable the dd command can be for said use case.

And, are there any more reliable and/or easier alternative?

9
  • 4
    I do not think dd is considered a good backup tool. It also is slow as it copies all empty space also. askubuntu.com/questions/2596/comparison-of-backup-tools I prefer just to use rsync, but copy to different devices at different times. help.ubuntu.com/community/BackupYourSystem & help.ubuntu.com/community/CategoryBackupRecovery
    – oldfred
    May 17 at 2:29
  • 11
    I think I read somewhere that dd does not check for or report errors Really!!?? Looks like fake news, about a coreutil piece of software. dd works well. But unless you are trying to build an exact image, I wouldn't call it a backup tool. May 17 at 3:29
  • I am trying to build an exact image. My goal here is to dd the SSD, nvme secure erase it, and then send the device to the service center. What I want know if I will be able to restore it later with no errors.
    – user472052
    May 17 at 3:58
  • 3
    I think I read somewhere that dd does not check for or report errors. You can verify this is not true from the documentation: As a simple rescue method, call dd as shown in the following example: the operand ‘conv=noerror,sync’ is used to continue after read errors and to pad out bad reads
    – Bert
    May 17 at 11:45
  • 1
    Note that it is not uncommon for manufacturers to send back to you a disk with characteristics not identical to the original one. Nowadays that's probably just a different slightly higher number of blocks so it shouldn't be much of an issue, but don't expect the disk to be exactly identical to the original one. IIRC there are a few partitioning or RAID schemes which are based on the end of the volume/partition, though.
    – jcaron
    May 17 at 13:58
27

TLDR: Use ddrescue

It supports resume/continue capabilities, has automatic logs, and tons of other options. More at the ddrescue home page.

Example syntax:

ddrescue /dev/sde yourimagename.image sde.log

IF you want to (given your comment mentioning restoring) restore the image from the command above onto another drive of the same exact size:

ddrescue -f yourimagehere.image /dev/sde restore.logfile

Furthermore, it is faster than dd is -- at least it does look like it is when comparing speed of ddrescue and dd + pv.

14
  • Does it work well with NVME drives? My goal here is to create an image of the SSD, nvme secure erase it, and then send the device to the service center. What I want know if I will be able to restore it later with no errors.
    – user472052
    May 17 at 4:02
  • 8
    Plain dd is fast if you use bs=128k or something, to make read/write system calls in 128kiB blocks instead of the default 512-byte sectors. (About half of your CPUs L2 cache size is a good tradeoff between per-system-call overhead vs. the kernel's copy_to_user / copy_from_user cache hits). Piping it through pv obviously costs more CPU time, although if you run pv to have it attack to an already running process (looking at file-positions in /proc/<PID>/fdinfo) then it avoids that memory bandwidth cost. I assume ddrescue chooses a reasonable block size by default. May 17 at 18:05
  • 2
    @PeterCordes OTOH, because of stuff like the -S option you can make ddrescue even faster than plain dd, as well as being able to do some other useful things like pre-allocating the output file or bypassing the OS’s page cache. May 17 at 18:08
  • 1
    If you just want progress stats, GNU dd status=progress shows what it's doing. You don't need pv for that. (And even on any Unix, you could just ls -l the output file whenever you want.) May 17 at 19:03
  • 1
    @AustinHemmelgarn: GNU dd has conv=sparse. IDK when that feature was added. It doesn't have a preallocate option; you have to use fallocate(1) and dd bs=128k conv=notrunc,sparse. Although I think sparseness detection works in units of the bs, so you might want a smaller block size like 16k. For a device that might have read errors, certainly ddrescue is a very good choice, but other than that you don't need it. If you know its options then by all means use it. Good point about tricks like sparsifying the output, especially if you just used fstrim on your FS before backup. May 17 at 19:33
5

The original comment about 'not check for or report errors' probably came because by default 'dd' does not pad out bad reads, so for block-orientated devices not only the bad block but all subsequent blocks will be incorrect (because they're no longer aligned). As others have said, this is fixable using the 'conv=noerror,sync' option, which tells dd to ensure blocks are remain on block boundaries. It should be joined by a block-size setting matching the filesystem block size, which is often 4096 bytes but can be lower.

I would agree with Johan Myréen's comment about using file backup, because the granularity of the backup is much smaller - an error backing up one file doesn't necessarily affect the others. You could also use a file system that uses error-correction on the file data (such as zfs, btrfs, and some configurations of others), so at the very least you know when errors happen and hopefully can fix them.

Another way to detect bad backups would be to use a message digest hash code, e.g. 'sha256' on the raw device (unmounted!!) and on the dd backup file... they should of course be the same.

Finally, best practice in backups is never to rely on only one backup... keep a minimum of 2!

1
1

A disk cloning utility - such as clonezilla will make a compressed copy of your disk, including partition tables etc but omitting unallocated space and also unused space in well-known file systems such as ext4 etc.

Obviously it is much faster to only copy data and not the unused space.

If you suspect that your NVME storage cannot be relied upon for read operations, then perhaps you might want to consider software such as rsync to extract most/all of your data (in addition to a cloning operation).

0

It is quite OK for whole of disk. You could pipe it through compression, and unused SSD space should compress well. Optimal block sizing helps.

However, compression does not work well with encryption.

Further, copying file system contents will provide a performance benefit and naturally create contiguous files, and while we may not think this matters for SSD, every little bit helps and contiguous free space helps.

.. and yes the performance benefit comes from not copying the space, or even the cluster tips.

ddrescue shines for magnetic media, if your SSD has errors, no amount of retrying is going to help, and there is no platter or moving head to settle from another direction. But ultimately you need to compare entire images (or their hash digest is often considered close enough) to verify.

0

dd, as the name suggests, makes a disk image backup. If you have a 1 TB drive with 100 GB used, the disk image will still be 1 TB. The conv=sparse option exists for dd if you know there are zero-blocks, and ddrescue has the --sparse command.

However, for backup purposes, you may not need a raw disk image. Copying files with rsync or putting everything into a (possibly compressed) tarball is sufficient for most backup use-cases, and working on a file level with a filesystem can be nicer than directly working with a mounted image. rsync also has a convenient delta-transfer algorithm which only sends the differences in files for updating backups (not the same as incremental backups, for which you'll need additional software).

4
  • "As the name suggests"? Out of curiosity: how does the name suggest anything? I've seen the one or the other d explained as "disk", "drive", "data", "dump", "destroyer". How to get from dd to " disk image backup" is a mystery to me. If I didn't know dd, its name would suggest me nothing. May 18 at 8:15
  • 1
    It was supposed to be a joke because dd famously has many nicknames but I guess the way I wrote it the joke doesn't come across at all
    – qwr
    May 18 at 9:09
  • The use case of the OP as described in the comments (sigh) is to produce a disk image. May 18 at 12:00
0

Have a look at partclone (apt-get install partclone), I think it is what you are looking for.

0

Yes, dd is fine but not really necessary.

I just use "cat" plus pbzip2 to create compressed images, e.g.:

# cat /dev/nvme0n1 | pv | pbzip2 > myimage.img.bz2
0

This hypocritic answer should have probably just been a comment.

Use rsync. It's the tool for the job.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.