6

I've been using dd in Linux to overwrite an external USB hard drive.

When I use the default block size of 512 bytes, with this command:

# dd if=/dev/zero of=/dev/sdb

throughout the whole operation, the hard drive (/dev/sdb) is being read from and written to alternately, approximately 1GB at a time. I.e. read 1GB...write 1GB...read 1GB...write 1GB etc. As much data is read from the hard drive as is written to it.

I know this is happening because it's showing in my custom Conky panel (diskio_read, diskio_write), which I know to be a 100% reliable indicator of disk I/O activity.

I've repeated this using a different external hard drive on a different computer. It happens via both USB 2.0 and USB 3.0.

In contrast, when I do the same thing, but use a block size of 1MB instead, with this command:

# dd if=/dev/zero of=/dev/sdb bs=1M

apart from a small amount of reading at the start, the hard drive is not read from at all during the operation.

Given that this phenomenon has happened on two different computers and two different hard drives of mine, using a standard Linux distro (Xubuntu 14.04), anyone who wants to should presumably be able to replicate it on their own computer.

Can someone please explain what is happening here?

Improve this question
5
  • Where do you see this? Please elaborate.
    – frostschutz
    Dec 3, 2015 at 17:13
  • Where/how do you observe those read/write? (iostat, strace, btrace)? I assume it's Linux, right? What does grep . /sys/block/sdb/queue/*size* give?
    – Stéphane Chazelas
    Dec 3, 2015 at 17:16
  • I've elaborated on how I know about the I/O. I can't read that file because I'm not overwriting the hard drive any more.
    – EmmaV
    Dec 3, 2015 at 17:32
  • And this, once again, shows why you should just use cat.
    – Gilles 'SO- stop being evil'
    Dec 3, 2015 at 23:45
  • @Gilles My intention was to write nulls through dm-crypt, so I expect there will be the same problem if I use cat.
    – EmmaV
    Dec 6, 2015 at 0:28

2 Answers 2

Reset to default
7

If you specify a block size (512 bytes) of less than the block size of the disk (often 4096 bytes, but nowadays maybe more), the block will be partially written, so that the contents of the rest of the block must be preserved before writing.

This is because disk blocks cannot be written to with only 512 bytes, but you have to write a full block at once (4096 or larger). When you write this (4096) amount or more, there is no partial write, so it does not have to read.

Improve this answer
0

This is basically a symptom of buffered I/O on a Linux system. A good overview of this in relation to the dd command is located at Is there a way to determine the optimal value for the bs parameter to dd? which supports ctrl-d's answer. However, this also suggests that even larger block sizes (i.e., greater than 1M) might improve performance further.

Improve this answer
1
  • Actually, the reads are not due to buffered I/O. They would be just as necessary if I/O was unbuffered.
    – Gilles 'SO- stop being evil'
    Dec 3, 2015 at 23:46

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .