10

From the Arch Linux Wiki: https://wiki.archlinux.org/index.php/USB_flash_installation_media

# dd bs=4M if=/path/to/archlinux.iso of=/dev/sdx status=progress && sync

[...] Do not miss sync to complete before pulling the USB drive.

I would like to know

  • What does it do?
  • What consequences are there if left out?

Notes

dd command used with optional status=progress:

tar -xzOf archlinux-2016-09-03-dual.iso | dd of=/dev/disk2 bs=4M status=progress && sync

Or using pv for progress

tar -xzOf archlinux-2016-09-03-dual.iso | pv | dd of=/dev/disk2 bs=4M && sync
13

The dd does not bypass the kernel disk caches when it writes to a device, so some part of data may be not written yet to the USB stick upon dd completion. If you unplug your USB stick at that moment, the content on the USB stick would be inconsistent. Thus, your system could even fail to boot from this USB stick.

Sync flushes any still-in-cache data to the device.

Instead of invoking sync you could use fdatasync dd's conversion option:

fdatasync

physically write output file data before finishing

In your case, the command would be:

tar -xzOf archlinux-2016-09-03-dual.iso | \
dd of=/dev/disk2 bs=4M status=progress conv=fdatasync

The conv=fdatasync makes dd effectively call fdatasync() system call at the end of transfer just before dd exits (I checked this with dd's sources).

This confirms that dd would not bypass nor flush the caches unless explicitly instructed to do so.

  • Thanks for your contribution, however I am not sure that this statement is correct The dd does not bypass the kernel disk caches when it writes to a device. When writing to a file (over the file system layer of the kernel), things are cached. However, I am concerned about writing to devices. Please provide a source for that statement if you can, because that is the linchpin of this question. If true, it would provide a valid reason for running sync after a dd-to-device operation. – Jonathan Komar Sep 28 '16 at 14:30
  • Yes it is cached. The caching happens inside block device infrastructure of the kernel. The file operations itself are not cached. the underlying block device interface does cashing. source: lxr.free-electrons.com/source/block/blk-flush.c – Serge Sep 28 '16 at 14:45
  • @macmadness86 see the updated answer – Serge Sep 28 '16 at 15:29
  • 5
    I preffer using oflag=sync, so progress outputs the real transfer speed and not the cached one (so going a steady 10MB/s instead of one second 100MB/s and then 10 seconds of stall). – Bart Polot Dec 21 '17 at 10:29
0

From the sync(1) manual page : "sync - Synchronize cached writes to persistent storage". Basically sync makes sure that all your data is written to the stick from the cache.

  • I don't see where in your question appears the part about the light being up. – schaiba Sep 27 '16 at 13:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.