4

Why is my sequential read speed so (comparatively) slow?

While CrystalDiskMark on Win10 reports around 5GB/s (for reading as well as writing), I just do not get close to that performance on Linux. (A copy&paste of several hundreds of GBs on Windows from/to the same drive averaged around 2,5GB/s, so I do not think CDM is far off from real values here.)

A simple dd if=/dev/nvme0n1 of=/dev/null bs=1M count=10k reports a mere 1.5GB/s.

On another NVME (both being Corsair Force MP600 1TB) dd reports 1.4 GB/s.

I would expect that such a sequential access is the best-case for reading from any storage device, so I really have no clue on what is going on here.

(I saw some similar questions on StackExchange, but they all went into different directions than this 'simple one'.)

Note aside: CrystalDiskMark uses 'real files' if I'm not mistaken - so it has even additional file system overhead, whereas my dd call should be the best one could possibly get - or not?

System info:

  • both NVMEs are connected with 4 PCIe 4.0 lanes
  • temperature of both NVMEs < 60°C
  • the faster one is also mounted as root, the slower one was unmounted
  • Zen2 Threadripper (so more than enough PCIe 4.0 lanes..)
  • Kernel 5.6.4
  • BIOS up2date
  • NVME firmware up2date

Any ideas or pointers into the right direction would be greatly appreciated!

2 Answers 2

3

The culprit is sequential access. NVMEs only show their performance on many simultaneous request. So a "cp" will simply result in one [sequential] read, as does dd and also hdparm. If you use tricks like "parallel" to create a cp process per file, the total throughput becomes a lot higher. Windows' Explorer seems to do just that even for big files (copying several segments in parallel - at least I guess so).

0

Be real, real careful with hdparm, read the man page, and one command that will help in Linux for read performance is:

sudo hdparm -tT --direct /dev/nvmexxx

The problem with measuring performance with dd is apparently how Linux pages in memory. Still trying to understand it fully myself. I suspect Windows has the same futz, it's just hidden better because theres no dd command. As I understand it, hdparm goes around the OS to measure the device performance much like your CrystalDiskMark does.

It's explained in the man page --direct option.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .