138

I've copied a large file to a USB disk mounted on a Linux system with async. This returns to a command prompt relatively quickly, but when I type sync, of course, it all has to go to disk, and that takes a long time.

I understand that it's going to be slow, but is there somewhere where I can watch a counter go down to zero? Watching buffers in top doesn't help.

Improve this question
1
235

Looking at /proc/meminfo will show the Dirty number shrinking over time as all the data spools out; some of it may spill into Writeback as well. That will be a summary against all devices, but in the cases where one device on the system is much slower than the rest you'll usually end up where everything in that queue is related to it. You'll probably find the Dirty number large when you start and the sync finishes about the same time it approaches 0. Try this to get an interactive display:

watch -d grep -e Dirty: -e Writeback: /proc/meminfo

With regular disks I can normally ignore Writeback, but I'm not sure if it's involved more often in the USB transfer path. If it just bounces up and down without a clear trend to it, you can probably just look at the Dirty number.

Improve this answer
1
  • 5
    On my system, writeback stays at a few megs until right near the end, when Dirty is empty, at which point it starts going down too. – mattdm Oct 8 '12 at 2:02
26

You can look at the /sys/block/<device>/stat file for the appropriate device while you're syncing. The 9th column will indicate the number of in-flight requests on the device, which should go down to zero when the sync is done.
Don't know of a way to translate that to a number of bytes, but it should give you a rough idea of how much "stuff" is still pending.

See the stat.txt file in the kernel documentation for a bit more information. (There's also an inflight file in that directory on my system which looks like it could contain read and write in-flight requests, but I can't find docs for that.)

Improve this answer
4
  • 3
    Just for handy reference, combining an idea from another answer: watch -t -n1 'awk "{ print \$9 }" /sys/block/sdd/stat' – mattdm Oct 8 '12 at 1:59
  • 6
    For my usb stick, this tends to hover around 150 throughout the duration of the copy operation and the sync afterwards. It does go to 0, but only at the very end. That makes the other answer more useful for impatiently watching progress. – mattdm Oct 8 '12 at 2:01
  • 1
    (Even though in theory I like looking at just the appropriate device rather than the systemwide info.) – mattdm Oct 8 '12 at 2:03
  • Is there a way to get amount of bytes written (synced, flushed not in flight) or cached to be written to drive? If I had a write-cache size per drive I could use that with du to calculate the real data that is synced on drive. – unfa Apr 26 '17 at 10:06
16

By using Greg's answer, you can simply have sync run in background while displaying the state of the Dirty block in memory.

To achieve this, simply run this command:

sync & watch -n 1 grep -e Dirty: /proc/meminfo

This will call sync in the background while executing watch in the front. When the sync command will have finished (around when the size of the Dirty block has reached 0), you will have an output that looks like this :

1]  + 27260 done        sync

This means that the command has finished and you can kill the watch command with Ctrl+C.

Improve this answer
3
  • 1
    BTW: I know this is an old question, but it is the first one that pops-up on Google when someone looks for it, and I wanted to add the solution I found. – jflemieux May 24 '17 at 14:15
  • 4
    Too bad sync/umount didn't get updates from the kernel and could print this kind of stuff directly if told to. – mcr Oct 7 '17 at 22:16
  • That's cute. Thanks for improving my surprisingly popular suggestion. – Greg Smith Feb 15 at 21:42
1

You can watch current block device I/O traffic with nmon, as in:

NMON=ld nmon -s1

(this preselects display of load graph and disk graph with 1 second refresh time.. this can also be set by starting nmon without parameters, than pressing l, d, - to toggle graphs and reduce refresh time from default 2s .. + of course would increase refresh delay.)

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.