Thanks to Anon's answer, I figured out that the file system was at fault - I used NTFS. The following are the results using FAT32.


diskspd64 -b128K -d5 -o32 -t1 -W0 -Sh -w0 cdm
508, 518, 520, 513, 513

fio --name=dontknow --ioengine=windowsaio --thread --size=1024m --bs=128k --time_based=1 --runtime=5s --iodepth=32 --numjobs=1 --rw=read --direct=1 --buffered=0 --startdelay=0s --filename=cdm
557, 557, 557, 558, 556


diskspd -b128K -d5 -o32 -t1 -W0 -Sh -w0 cdm
529, 528, 529, 529, 529

fio --name=dontknow --ioengine=libaio --thread --size=1024m --bs=128k --time_based=1 --runtime=5s --iodepth=32 --numjobs=1 --rw=read --direct=1 --buffered=0 --startdelay=0s --filename=cdm
560, 560, 560, 560, 559


Based on the same input file on the same drive, these are the resulting numbers for read speed (MB/s - I ran each 5 times) for given commands on Windows:

diskspd64 -b128k -d5 -o32 -t1 -W0 -S -w0 cdm
555, 555, 556, 556, 555

fio --name=doesntmatter --ioengine=windowsaio --thread=1 --size=1024m --bs=128k --time_based=1 --runtime=5s --iodepth=32 --numjobs=1 --rw=read --direct=1 --startdelay=0s --filename=cdm
561, 553, 562, 561, 558

And Linux (to be precise - KDE neon useredition-20180802):

diskspd -b128K -d5 -o32 -t1 -W0 -Sh -w0 cdm
1800, 2000, 1925, 1891, 1973

fio --name=doesntmatter --ioengine=libaio --thread=1 --size=1024m --bs=128k --time_based=1 --runtime=5s --iodepth=32 --numjobs=1 --rw=read --direct=1 --startdelay=0s --filename=cdm
2637, 2826, 2593, 2770

I would also like to mention that this is a SATA SSD drive with an official maximum read speed of 555 MB/s. So the Windows numbers seem to be accurate.

  • Weird! iotop should measure the bandwidth of uncached IOs, I wonder if it shows more realistic figures than the statistics at the end of the fio run. – sourcejedi Aug 8 '18 at 18:22
  • It doesn't make sense for me to use other tools unless they can be similarly fine-tuned as diskspd and fio and produce similar results. Because somebody needs to finally develop a cross-platform alternative to CrystalDiskMark and that somebody might be me. That is - IF someone can answer my question... – AndyO Aug 8 '18 at 19:24
  • Sorry, I mean to try and illuminate the surprising results you're getting. Not as an alternative to fio. – sourcejedi Aug 8 '18 at 20:03
  • Ideally I would use sync=1, or a longer test to avoid caching effects. But direct IO should still have to traverse the SATA bus, limited to an absolute max of 600 MB/s, so I don't think that explains it. – sourcejedi Aug 8 '18 at 20:18
  • It's all good - I simply explained why that unfortunately doesn't help me. :) – AndyO Aug 8 '18 at 20:55

Unfortunately there's not enough information to answer your question - it is often necessary to see the full fio output from your run and to know what version of fio you are running because this can say things like what depths are achieved and how busy Linux thought the disk was during the run (e.g. when latencies are close to 0 that's almost always a sign of caching taking place).

Could be that the filesystem the file is on doesn't support direct=1 with the options it is using. Could be your file was entirely cached for some reason and you're just reading back from the cache (watch out for this when file sizes are dramatically less than your total RAM). Could be that because you didn't write to your file it's sparse/empty and not really "there" (try doing a full set of writes before you read it back)...

PS: thread doesn't need to take a value (see http://fio.readthedocs.io/en/latest/fio_doc.html#cmdoption-arg-thread ).

  • Thank you very much! It was the file system! Wish I would've realized this before I blindly ordered a new drive... (Probably would've anyway though) – AndyO Aug 8 '18 at 23:36
  • 2
    You're welcome! Thanks for diligently updating your question for the next person... – Anon Aug 9 '18 at 8:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.