0

According to articles on the web, mount with nobarrier will speed the disk up:

  1. Write small blocks of data to img (with barrier): SLOW

    # dd if=/dev/zero of=xfs.img bs=1024 count=20000
    # mkfs.xfs xfs.img
    # mkdir -p xfs
    # mount -o loop xfs.img xfs
    # dd if=/dev/zero of=output bs=32K count=1 conv=fsync
    32768 bytes (33 kB) copied,0.01037167 s, 2.4 MB/s
    
  2. Write small blocks of data to img (-o nobarrier): FAST

    # dd if=/dev/zero of=xfs.img bs=1024 count=20000
    # mkfs.xfs xfs.img
    # mkdir -p xfs
    # mount -o loop,nobarrier xfs.img xfs
    # cd xfs ; dd if=/dev/zero of=output bs=32K count=1 conv=fsync
    32768 bytes (33 kB) copied, 0.000608567 s, 53.8 MB/s
    

Now, I want to remount my / to add the nobarrier flag. So I edited /etc/fstab:

/dev/sda2      /      xfs     defaults,nobarrier    0    0

then mount -o remount /.

But the result is not good:

# pwd
/root
# dd if=/dev/zero of=output bs=32K count=1 conv=fsync
32768 bytes (33 kB) copied, 0.00811443 s, 4.0 MB/s

I cannot understand why nobarrier is working for dd-img, but not working for existed partitions. Can anyone tell me?

0

When using a loopback filesystem, the filesystem it is created on must be taken into account. Specifically, the kernel will likely not be immediately flushing the written pages of the loopback FS to the containing filesystem's disk, despite fsync being called on the loopback fs. Those writes may be syncd on the loopback, but they may be sitting as dirty pages in memory for the containing FS.

Now, how the nobarrier option interacts with the loopback driver, as well as the containing filesystem, is something I'm not sure of. As such, I did an experiment. I added the variable of mounting the containing filesystem with sync. The results are as follows.

(all outputs from dd if=/dev/zero of=xfs/output bs=32K count=10000 conv=fsync)

  1. containing fs async, loopback barrier

    32768000 bytes (33 MB) copied, 0.401873 s, 81.5 MB/s
    
  2. containing fs async, loopback nobarrier

    32768000 bytes (33 MB) copied, 0.0414423 s, 791 MB/s
    
  3. containing fs sync, loopback barrier

    32768000 bytes (33 MB) copied, 71.5749 s, 458 kB/s
    
  4. containing fs sync, loopback nobarrier

    32768000 bytes (33 MB) copied, 70.6415 s, 464 kB/s
    

The barrier and nobarrier speeds vary heavily when the containing fs is async. However when the containing fs is sync and doesn't benefit from the page cache, the speedup mostly disappears.

The conclusion is that testing barrier and nobarrier with a loopback filesystem isn't going to be helpful. The containing filesystem's interaction with the kernel caching is going to get in the way. I suspect that the kernel cache is not the only thing which will cause erroneous results when testing performance using loopbacks.

| improve this answer | |
0

Be careful with such assumptions on Linux. Linux was optimized to create the impression of speed with respect to filesystems instead of optimizing the real speed.

The reason for getting the impression of high speed is that the file data (in many cases) is not on the disk when the writing program finishes. In other words, you cannot tell what you metered with your test as you do not know the filesystem state after dd finished.

If you like to get comparable results, you need to set up a performance test that is immune against the named problem, so you would need to write a file that is at least twice the size of your local RAM in the machine.

| improve this answer | |
  • 1
    There are a few ways to get around the caching stuff which are preferable to using a file larger than RAM. Using dd with fsync, like the asker is doing, will cause an fsync which forces the write to disk after each write. Mounting a filesystem in sync mode will accomplish the same thing. Also, testing directly against a block device without a FS will avoid any caching layer. Alas, in the asker's case the loopback usage is going to throw results off. – alienth Sep 17 '15 at 17:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.