3

I am testing my new PCI-E SSD in Linux.

I am using the following command to test its performance (reference: https://www.thomas-krenn.com/en/wiki/Linux_I/O_Performance_Tests_using_dd)

(1) dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=2048 --> 2.2GB/sec

(2) dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=2048 oflag=direct --> 2.2GB/sec

(3) dd if=/dev/zero of=/mnt/nvme0n1/tempfile bs=1M count=2048 --> 80MB/sec

(4) dd if=/dev/zero of=/mnt/nvme0n1/tempfile bs=1M count=2048 oflag=direct --> 800MB/sec

My guesses are as follows: (3, 4) is writing over the filesystem (formatted as NTFS for some reasons). However, (1, 2) is writing into the block device directly, which gives no overheads of file system.

Am I correct or not? Can you give me some explanations on it?

Thanks

  • 2
    mostly. im not sure about writing over the filesystem - it writes into the filesystem. this is probably especially true of NTFS on a linux system - which would be FUSE and so a terrible control for a benchmark. – mikeserv Dec 6 '15 at 8:23
  • I do agree with mike. NTFS over FUSE is not suitable for your purposes. – Rui F Ribeiro Dec 6 '15 at 8:36
  • Can you run your tests with the conv=fdatasync option? Without it, you're not getting a complete picture of how long the i/o takes. – Mark Plotnick Dec 6 '15 at 10:21
4
  1. I would quibble over your wording.  I would say that commands (1) and (2) are over-writing the filesystem, if any; i.e., ignoring it and destroying it (if there is one).  They will behave the same if the device has a filesystem on it before hand or not.

    Meanwhile, commands (3) and (4) are writing into the filesystem, or through it.

  2. Yes, of course the fact that commands (3) and (4) are going through the filesystem code is why you get the performance difference.  (Continued in paragraph 4.)

  3. I don't see why the fact that the filesystem is NTFS is really significant.  I suspect that you would get similar results with any filesystem type; e.g., one of the ext family.

  4. To build upon point 2: First of all, filesystem I/O largely ignores bs=anything and oflag=direct.  The filesystem code probably treats a write of 1M as 2048 writes of 512 bytes, or perhaps 256 writes of 4K bytes.  Secondly, the filesystem code has the job of maintaining filesystem integrity.  That means, each time it extends your tempfile, it must allocate blocks from the free list and allocate them to the file.  This means it must be continually modifying the free list and the file's inode (or equivalent on whatever the filesystem type is).  This means not only (potentially) three actual writes for every user-visible write, but that the writes would be non-contiguous, and the drive would be seeking all over the place.  Further, if the filesystem has been in use for a while, the free list may have gotten out of order, and so the blocks allocated to the file may be non-contiguous.

Suggestions:

  • Do tests (3) and (4) after a mkfs, so the filesystem is clean.  Then repeat them with the file already existing.  This should decrease the amount of bookkeeping I/O.
  • Repeat all the tests with bs in the 512-4K range.  The results for tests (3) and (4) should be nearly unchanged, while those for tests (1) and (2) should be much lower.
| improve this answer | |
  • See the comments about fuse/ntfs. Adjust your answer to it and it will be perfect. – Nils Dec 6 '15 at 9:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.