3

I have made my peace with the fact that NTFS(-3g) on Linux will be slower than NTFS performance on Windows. I can write to my external NTFS-formatted USB 3.0 HDD at about 100+ MB/s on Windows while I have to settle for 30 MB/s (give or take) on my Debian (Wheezy) box.

That's not the problem, however. I found (empirically) that if I want to copy, say, 20 files from my box to the HDD, the copying starts off at the "normal" 30 MB/s but gradually slows down to a miserly 4 MB/s!! Whereas, if I do the copy, say, 5 files at a time (serially) the copying speed remains at 30 MB/s for all four copy processes. This is not specific to Debian, by the way. I've observed similar behavior on Fedora and Ubuntu.

My question is, is this behavior normal or should I be concerned? If something is wrong, how should I go about debugging it/fixing it?

3

You are seeing the effects of drive head seek latency when running the parallel copies.

With most file systems, including NTFS and ext[234], data is stored in distinct locations on the drive. File system information here, block allocation data over there and file data way over there.

When writing a single file, the metadata changes relatively infrequently, so the head is mostly in the right place to be writing data blocks. When running 20 concurrent writes, the head has to dash between the block allocation areas and the data areas about 20 times as often and disk head seeks are measured in the 10s of milliseconds.

When writing to a native filesystem, some liberties can be taken with the amount of seeking done (for example, keeping a copy of part of the free-list in memory and only writing that out infrequently, thus saving a bunch of seeks). I expect the same applies to NTFS under Windows, but the NTFS Linux filesystem developers can't afford to be as cavalier, opting for consistency over performance.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.