8

I want to know why systems become slow when writing mass data to disk.

I think that for system to become slow, there should be some issue with CPU. But write is only I/O bound.

Do hardware interrupts occur when writing data? If so, it may be because of the interrupts that the CPU is always context switching.

  • 1
    I think almost all applications will read/write data from disk, isn't it? – jilen Jun 8 '13 at 12:18
  • 1
    Maybe it is swapping to memory and therefore slowing down, when the disk is under heavy use otherwise. A systemload-plugin could tell you, whether you're using swap in a graphical way. For the commandline, use free. – user unknown Jun 8 '13 at 12:20
  • 1
    If you have sysstat configured/running you can look at the SAR reports for the times that are slow and it will show you what is happening around that time (context switches, Disk I/O, CPU load, network traffic, etc). – Bratchley Jun 8 '13 at 12:56
  • Also have a look at your $PATH setting(s) if you use command-completion or make a lot of spelling mistakes. Looking through many directories, especially ones with many directory entries, can take time which is noticeable when resources are scarce. – MattBianco Apr 23 '14 at 11:55
3

The core reason behind is that the usual: I/O is much slower than CPU/RAM. Even if the processes doing I/O operations use DMA (which offloads the CPU), at some point they are likely to need to wait on the completing of their requests.

In the most usual case of a HDD just add in several applications trying to access files scattered around the drive, and you can make yourself a coffee (tea, whatever). With SSDs the situation gets better, but even an SSD - which has throughput measured in hundreds of MB/s on SATA (as compared to tens of MB/s of a spin-plate HDD) and really negligible seek times (compared to miliseconds for a spin-plate) - can become a bottleneck.

The problem as I understand it is not just in the data transfers themselves, but in the necessary overhead - I/O is controlled by kernel, but only seldom happens without userspace. Thus there can be a plenty of context switches, just from the applications waiting on I/O checking whether something is happening (depends on implementation, of course). In the case of disk transfers, there may well be several kernel threads competing for resources or busy-waiting (which sometimes is appropriate strategy). Remember, for example copying data from one partition to another requires a modern filesystem to: find out where the source data is, read it, allocate space on the target file system, write meta data, write data, repeat until finished.

And if, at some point, your system starts swapping (which usually has higher priority than regular I/O), the disaster is finalized.

EDIT: After talking to some Linux kernel developers the situation became a bit clearer. The main problem is the I/O scheduler, which doesn't have much idea about which I/O to prioritise. Hence any user input and following graphical output is sharing the queue with the disk/network activity. As a consequence of that, it may also happen that it may throw away cached process data from page cache (e.g. loaded libraries) when it concludes it can use the page cache more effectively on other I/O. That of course means that once that code needs to be run again, it will have to be fetched again - form the disk that may be under heavy load already.

That said, as far as the Linux kernel goes, many of these issues have been fixed recently (the problem has been known), so say 4.4.x or 4.5.x should behave better that it used to and problems should be reported (generally the kernel people are happy when someone wants to help by bug reporting and testing).

  • Could you elaborate a bit, why the overall system lagging when IO is heavily busy? For example, I don't see any connection between Windows Manager and IO — obviously, WM doesn't do any IO, so why would it start lagging (at least, when it isn't swapped)? – Hi-Angel Mar 4 '16 at 3:56
  • WM can actually be doing a lot of I/O via X server/Wayland compositor (not as much as copying four data streams between 2 HDDs, but it doesn't have to be completely negligible - which is probably one of the reasons awesome seems to be working a bit better than kwin in this sense). – peterph Mar 4 '16 at 21:08
  • But what kind of IO does it do? Is it simply because of logging into xsession-errors/Xorg.ₙ.log? Isn't the logging just buffered, without interrupting to WM work? Anything else? UPD: I just looked into the /proc/AwesomePID/fd — the only file my WM have opened is xsession-errors. Everything else not really a file — sockets, /dev/null, /proc/stat – Hi-Angel Mar 5 '16 at 6:18
  • 1
    But it is you said that WM can actually be doing a lot of I/O via X server/Wayland compositor. But whatever, why do you think that it doesn't get enough CPU. When it happens, CPU is usually pretty free — I can see the load in CPU widget on my panel, and unpacking an archive doesn't take much of it. – Hi-Angel Mar 6 '16 at 3:44
  • 1
    @Hi-Angel check update answer whether it clarifies things a bit. – peterph Mar 17 '16 at 12:20
2

My experience is that I/O activity alone does not slow down a system. This effect occurs when other tasks need I/O, too. The situation will become really evil if the system is swapping (forced to) and you cause heavy I/O load then.

You can influence the impact of I/O heavy tasks by ionice. If you put them to idle priority then the latency for other tasks may still increase but not beyond the minimum. The I/O task gets interrupted immediately if another (a non-idle) task has I/O to do. If you are using a scheduler which supports these settings.

See Selecting a Linux I/O Scheduler

  • 1
    My experience is the opposite: i.e. I may start unpacking a big archive — causing IO load — and attempting at this moment to switch to another «desktop», I feel lags of the switching. How is window manager even relevant to IO?? And yeah, I have a free RAM, so it's not swapped (and even if I haven't — wouldn't WM swapped in the last turn?). FWIW, I am using a lightweight Awesome WM, and earlier with kwin things were even worse. – Hi-Angel Mar 4 '16 at 3:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.