1

I have a computer which is experiencing some serious I/O issues.

Software: Kali-rolling, Linux x86-desktop-1 4.12.0-kali2-amd64 #1 SMP Debian 4.12.13-1kali2 (2017-10-03) x86_64 GNU/Linux. But I've had various kernels, stock ones, and custom ones.

Hardware: The relevant hardware is as follows. Ryzen 1800X CPU, 16 GiB RAM, MSI X370 SLI PLUS with the latest BIOS (version 3.6, but I've also tried three others), M.2: Crucial® MX300 525GB M.2 SSD and 256 GB Samsung SSD 840 PRO.

The problem: The problem can perhaps be best described just from a user perspective at first:

I'm copying a lot of small files, such as the Linux kernel source tree. This results in so much slowdown, that even a simple command such as the "ls" binary, or "dmesg" can take 15 seconds or more to run. The whole system freezes in terms of I/O, anything that requires any form of I/O will block until whatever is blocking it allows it to get through the I/O queue.

I've noticed the problem when doing the following:

  • Playing a video file over the network, from a CIFS resource. This form of I/O can introduce a second or two delay in the video streaming.
  • Playing an MP3 file over the network, while using a web browser can make the audio stutter.
  • Copying something locally on the SSD or M.2 device, perhaps especially when it involves many small files.

The problem started since I installed the OS and has always been there.

My reasoning and how I've tried to solve this: The hardware should be more than capable of handling multiple I/O requests at the same time. The idea that this would all be caused by a faulty SSD/M.2 seems unreasonable, as I've tried both an SSD, and a physically separate M.2 device, they both have the same problem. Furthermore, it also seems unreasonable that the stock kali kernel would have this particular I/O issue, especially since I've tried multiple kernels: 4.9, 4.12, 4.13.2, 4.13.10. I've considered that perhaps there are BIOS options that would affect this performance, but I couldn't find anything, regardless, I'm running setup defaults and have tried multiple BIOS versions, including version 3.4, 3.5, and 3.6.

I've checked dmesg for any I/O errors, there are none.

I've also considered that there are multiple I/O schedulers for Linux:

$ cat /sys/block/sda/queue/scheduler 
noop deadline [cfq] 

The scheduler can be changed by issuing e.g.

echo "noop" > /sys/block/sda/queue/scheduler

See this question for more information: https://stackoverflow.com/questions/1009577/selecting-a-linux-i-o-scheduler

However, I've tried noop, deadline, and CFQ, they don't appear to affect the problem at all.

I also wanted to check the performance of the non-volatile devices:

hdparm -t /dev/sda

/dev/sda:
 Timing buffered disk reads: 1112 MB in  3.01 seconds = 369.69 MB/sec
hdparm -t /dev/sda

/dev/sda:
 Timing buffered disk reads: 1122 MB in  3.00 seconds = 373.53 MB/sec

The performance is nothing staggering, I should actually have more than this. But even with this type of performance, these issues should be non-existent. I'm not sure if the issue I'm having is related to this poor I/O performance.

When I was running Gentoo on this same system, I got this on the 840 PRO device (hdparm -t):

510.82 MB/sec
524.05 MB/sec

How should I go about debugging this issue? Is it obvious to anyone what the problem is?

  • I'm experiencing similar problems. I've switched to deadline, which seemed to help a bit, but not much. Then I discovered that echo 3 | sudo tee /proc/sys/vm/drop_caches nearly always will unfreeze the blocked processes, sometimes after a bit of delay, and restore performance. I suspect multi-CPU deadlock in the cache structures, but that's just a guess. Don't know if it will help in your case, but it's worth a try. – dirkt Dec 2 '17 at 17:57
0

It does not explain what is happening but you could try to work around the problem by using ionice for drive operations and tc (traffic shaping) for network operations.

  • Since this is a highly interactive desktop system I don't think this is a viable option. Even if it would work, it wouldn't be possible for me to run everything with ionice. – AlphaCentauri Dec 3 '17 at 20:02
  • @AlphaCentauri It may help understand the problem better. And you probably would have to run very few processes with different settings only. – Hauke Laging Dec 3 '17 at 20:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.