Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I will be backing up a large (750GB) disk to an external USB disk using dd.
Should I be using redirection or piping? Which is more efficient? Or is there a difference?

Also, what is the best block size? USB is likely to be the bottleneck here.

dd if=/dev/sda bs=1M | gzip -c > /mnt/sdb1/backups/disk.img.gz

gzip -dc /mnt/sdb1/backups/disk.img.gz | dd of=/dev/sda bs=1M

vs

dd if=/dev/sda bs=1M | gzip -c | dd of=/mnt/sdb1/backups/disk.img.gz

dd if=/mnt/sdb1/backups/disk.img.gz | gzip -dc | dd of=/dev/sda bs=1M

Thanks.

share|improve this question

migrated from stackoverflow.com Mar 12 '12 at 11:44

This question came from our site for professional and enthusiast programmers.

2  
I would try to backup file systems, not raw disk images. And to tune the parameters, you'll have to benchmark. –  Basile Starynkevitch Mar 12 '12 at 6:20
    
dd is archaic (any other tool thankfully does away with bs= and count=, like ddrescue), and in your case also pretty much pointless. Just pipe it into gzip -c </dev/sda >disk.img.gz & gzip -cd disk.img.gz >/dev/sda. –  jørgensen Mar 12 '12 at 11:12
    
Or do it with rsync. Just construct the command with the parameters, and it will do you everything in the most efficient way it thinks. –  vakufo Mar 12 '12 at 11:53
    
In the second command of the second listing, what is the dd if=.../disk.img.gz | part supposed to do? As far as I understand, gzip -dc (=zcat) doesn't care for its stdin when there's a file argument. –  sr_ Mar 12 '12 at 13:57
    
@sr_ That was a typo. Fixed. Thanks. –  mcu Mar 12 '12 at 20:43

2 Answers 2

up vote 2 down vote accepted

Piping involves one more process and one more user-land copy, so it should be more efficient to use redirection.

But I guess that on nowadays hardware & software caching system, it should not make any real difference. Maybe you can have better results using bs=4k or bs=64k, since it's the pipe's limit under linux. See this question for more detail about different bs parameters.

share|improve this answer

You don't need to use dd or piping at all.

</dev/sda gzip -c >/mnt/sdb1/backups/disk.img.gz
</mnt/sdb1/backups/disk.img.gz gzip -dc >/dev/sda

I once made a benchmark and found using dd slower than cat for a straight copy between different disks. I would expect the pipe to make any solution involving dd even slower in this case.

share|improve this answer
    
That is an example of using redirection. –  psusi Mar 13 '12 at 17:42
    
@psusi Gah! I meant to write “dd or piping”. Thanks. –  Gilles Mar 13 '12 at 17:47

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.