27

From the Arch Linux Wiki: https://wiki.archlinux.org/index.php/USB_flash_installation_media

# dd bs=4M if=/path/to/archlinux.iso of=/dev/sdx status=progress && sync

[...] Do not miss sync to complete before pulling the USB drive.

I would like to know

  • What does it do?
  • What consequences are there if left out?

Notes

dd command used with optional status=progress:

tar -xzOf archlinux-2016-09-03-dual.iso | dd of=/dev/disk2 bs=4M status=progress && sync

Or using pv for progress

tar -xzOf archlinux-2016-09-03-dual.iso | pv | dd of=/dev/disk2 bs=4M && sync
Improve this question

2 Answers 2

Reset to default
45

The dd does not bypass the kernel disk caches when it writes to a device, so some part of data may be not written yet to the USB stick upon dd completion. If you unplug your USB stick at that moment, the content on the USB stick would be inconsistent. Thus, your system could even fail to boot from this USB stick.

Sync flushes any still-in-cache data to the device.

Instead of invoking sync you could use fdatasync dd's conversion option:

fdatasync

physically write output file data before finishing

In your case, the command would be:

tar -xzOf archlinux-2016-09-03-dual.iso | \
dd of=/dev/disk2 bs=4M status=progress conv=fdatasync

The conv=fdatasync makes dd effectively call fdatasync() system call at the end of transfer just before dd exits (I checked this with dd's sources).

This confirms that dd would not bypass nor flush the caches unless explicitly instructed to do so.

Improve this answer
11
  • 1
    Thanks for your contribution, however I am not sure that this statement is correct The dd does not bypass the kernel disk caches when it writes to a device. When writing to a file (over the file system layer of the kernel), things are cached. However, I am concerned about writing to devices. Please provide a source for that statement if you can, because that is the linchpin of this question. If true, it would provide a valid reason for running sync after a dd-to-device operation.
    – Jonathan Komar
    Sep 28, 2016 at 14:30
  • Yes it is cached. The caching happens inside block device infrastructure of the kernel. The file operations itself are not cached. the underlying block device interface does cashing. source: lxr.free-electrons.com/source/block/blk-flush.c
    – Serge
    Sep 28, 2016 at 14:45
  • 12
    I preffer using oflag=sync, so progress outputs the real transfer speed and not the cached one (so going a steady 10MB/s instead of one second 100MB/s and then 10 seconds of stall).
    – Bart Polot
    Dec 21, 2017 at 10:29
  • 1
    @JonathanKomar, @DJCrashdummy, yes, that's why both options have absolutely the same effect when we are writing directly to a block device: there is no fs metadata to sync. As for oflag=direct - it's interesting, I will check...
    – Serge
    Jan 23, 2022 at 9:44
  • 1
    @Serge Any update with oflag=direct?
    – Jonathan Komar
    Apr 26, 2022 at 19:42
1

From the sync(1) manual page : "sync - Synchronize cached writes to persistent storage". Basically sync makes sure that all your data is written to the stick from the cache.

Improve this answer
1
  • I don't see where in your question appears the part about the light being up.
    – schaiba
    Sep 27, 2016 at 13:25

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .