7

Every time I've written a file to an empty raw block device, e.g.

# dd if=image.iso of=/dev/sdb status=progress

I've never used any type of sync (i.e. sync; conv=fsync; conv=fdatasync; oflag=sync; oflag=dsync).

I've noticed that dd doesn't ever exit until all writing has finished.

I always verify this using Conky's I/O facility and grep Dirty /proc/meminfo. Also, the checksum of the device always matches that of the file that was written to it. So I am always 100% sure that the whole file was written to the device.

I've written files to an ext4 volume to compare. For example using:

$ dd if=/dev/urandom of=~/file bs=1M count=50 iflag=fullblock

When writing to an ext4 volume, after dd exits there is always a delay of about 20 seconds before the data is actually written to the disk.

Many people advocate using the sync command after the dd command, or including one of the several sync options in the dd command when writing to a block device. E.g. here and here. However, I've not known anyone actually prove that it is necessary.

One of the comments on this page is:

sync is pointless here [i.e. writing directly to /dev/sdX]. It only affects file system operations.

Five people have upvoted this comment and it is consistent with my experience.

So when writing to a block device, are the any circumstances in which dd will exit before all writing has completely finished? Has this actually ever happened to anyone?

What about other writing options, such as cp and cat? Can they exit before writing to a block device has finished?

  • 1
    @GAD3R: By 'any type of sync' I mean the sync command and all dd's sync options. – EmmaV Oct 9 '19 at 13:39
  • According to the man page, sync flushes filesystem metadata and cached file data to the underlying filesystem. When you're dealing with the device directly, there is no "filesystem". I suspect a sync after a dd is unnecessary. – Andy Dalton Oct 9 '19 at 14:00
  • @AndyDalton, that's assuming the man page is technically exact. Is the text there enough to prove that the system doesn't do write caching on block devices? – ilkkachu Oct 9 '19 at 14:23
  • @ikkachu No, and that's why I expressed that as a comment and not an answer. – Andy Dalton Oct 9 '19 at 14:53
  • unix.stackexchange.com/questions/495670/… This Q 8 months ago got no reaction. – rastafile Oct 10 '19 at 9:14
-3

This is the first thing I read with info sync, which takes me to a section "Synchronizing I/O operations" (arch linux).

In most modern operating systems, the normal I/O operations are not executed synchronously. I.e., even if a 'write' system call returns, this does not mean the data is actually written to the media, e.g., the disk.

With man 3 sync I get "POSIX Programmer's manual": The function sync() shall...schedule.... And then:

The writing, although scheduled, is not necessarily complete upon return from sync().

I see two situations where that matters:

programmers: If you do a low level read immediatly after a write (something like that)

users: If you detach the (external) block-device immediately after the write. If you don't umount, then a sync might help. But if you umount anyway (and you should), you don't need a sync.

I guess it makes a difference in practice if you do a single dd with small blocks to a fast internal device on an idle system (no way for a user to disturb that) or if several dd processes write huge blocks to diffrerent partitions of a slow USB.

Even with sync (man 1 sync):

BUGS Persistence guarantees vary per system. See the system calls below for more details.

The Q:

So when writing to a block device, are the any circumstances in which dd will exit before all writing has completely finished? Has this actually ever happened to anyone?

Well yes, if the lights are blinking that means USB flash still needs power to store away these blocks. And by the time you have finished typing "sync", it is done.

  • so the answer is: if the LED blinks then yes otherwise no. LOL – Eric Jan 6 at 12:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.