Questions tagged [io]
I/O is an acronym for input/output, which refers to the mechanism or process for the transfer of information between one system or component and another.
634
questions
1
vote
1
answer
44
views
How to find the parent process of child processes causing heavy IO
I would like to find the parent process for a large amount of small child processes performing small IO operations.
For example, consider the following python script:
import os
import time
import ...
- 233
0
votes
0
answers
24
views
How can I get the _real_ progress of dd, ignoring fast/staging buffers? [duplicate]
I'm using the dd command to write to a USB stick. My command is pretty straightforward:
dd if=myimage.iso of=/dev/sdd bs=1M status=progress
and indeed, I seem to be getting the progress reported:
...
- 9,808
0
votes
1
answer
62
views
Is my SSD dead? can't mount. I/O errors, 29k+ fsck errors
So I work with a couple of people and we ssh into a Linux server for work. There are several SSDs, and one of the non-boot Samsung 870 SSDs crashed the other day in the morning. This led to an ...
1
vote
2
answers
40
views
How to limit btrfs-send throughput?
I am trying btrfs send/receive a 3TB btrfs volume.
When I sent it to another machine over network, it went fine. The speed was like 20-30 MB/s.
However, when the destination disk is directly attached ...
- 377
4
votes
0
answers
47
views
What does it mean that /sys/block/sda/stat shows 'write I/Os' nonzero values, but 'write sectors' is 0 value
This is the output of cat /sys/block/sda/stat :
3227140 166605 7596010579 28294594 51 0 0 656 2 15228780 28295907 0 0 0 0 51 656.
Column 5 is 'write I/Os' currently 51, column 7 is 'write sectors' ...
- 41
0
votes
1
answer
97
views
cgroups v2 : io.latency not exists
When I create a cgroup using sudo mkdir /sys/fs/cgroup/my-cgroup, io.latency is not created. Other io files are created, like io.max, io.stat, io.pressure...
The content of cgroup.subtree_control ...
- 1
0
votes
1
answer
202
views
io_uring with `fio` fails on Rocky 9.3 w/kernel 5.14.0-362.18.1.el9_3.x86_64
I've tried various variations of the command:
fio --name=test --ioengine=io_uring --iodepth=64 --rw=rw --bs=4k --direct=1 --size=2G --numjobs=24 --filename=/dev/sdc
lower queue depth
direct set to 1/...
- 677
11
votes
2
answers
1k
views
Differences Between `/dev/null` and Devices Under `null_blk` Driver
I recently encountered the Linux Null Block device driver, null_blk, while I benchmarking the I/O stack instead of on a specific block device. I found the devices created under this driver (let's use ...
- 147
3
votes
1
answer
70
views
Why is Tar speed not same?
I heard somewhere on the Internet that cpio is faster than tar.
So I tried to check it with my shell script.
#1/bin/bash
rm ct.tar ct.cpio
sync && sudo sh -c "/usr/bin/echo 3 > /proc/...
- 31
6
votes
1
answer
60
views
How useful is ionice tool with modern linux kernel
The ionice tool is supposed to offer relief on high I/O load by executing commands only when the system is in a specified state.
The man page states:
Linux supports I/O scheduling priorities and ...
- 173
1
vote
0
answers
22
views
Determining cause of recurring external SSD data transfer with goal of preventing it
The LED lights for an external SSD connected via USB to a Linux Mint box indicate data transfer every 00:06:15 (6 mins 15 secs), despite no user applications (and possibly no processes) accessing the ...
2
votes
1
answer
46
views
GNU parallel: why does diagnostic output look like sequential execution rather than parallel execution?
Scenario:
$ cat libs.txt
lib.a
lib1.a
$ cat t1a.sh
f1()
{
local lib=$1
stdbuf -o0 printf "job for $lib started\n"
sleep 2
stdbuf -o0 printf "job for $...
- 629
0
votes
1
answer
107
views
Running a screensaver on top of console program after inactivity: how to watch for input?
I'm implementing a screensaver system that would wait for user input on a tty, and, in the event that it didn't get any, would play a screensaver until input is received.
How would I efficiently tap ...
- 115
2
votes
1
answer
68
views
How can a different controller explain this I/O?
My new NAS has 5 disks. They are all same model.
sde is connected to the controller on the motherboard. sda-sdd is on the raid controller.
Run 'pv /dev/sd[cde]' in parallel:
Device rkB/s wkB/...
- 36k
1
vote
0
answers
47
views
How to tune BFQ schduler to prevent a single low priority process from severely degrading system performance?
I have been trying to use nice/ionice for a while and it turns out that bfq ignores ionice parameters. So how do I prevent a low priority process from consuming all the io bandwith to the detriment of ...
- 11
1
vote
0
answers
135
views
Determining system's unresponsiveness when high IO load
I'm trying to understand why my Fedora 38 (KDE on Wayland) becomes unresponsive for some seconds while there's high IO on the root disk (a SanDisk SSD PLUS 240GB (UF5000RL)).
The root partition is ...
- 81
0
votes
0
answers
105
views
how to check if the IO was hanging
Are there any method I could use in centos to check if the block device under the IO was hanged?
I tried to check the /proc/diskstats, but it is not obvious enough to show if there is an hang there.
...
- 1
0
votes
1
answer
367
views
FIO reports slower sequential read than the advertised NVMe SSD read bandwidth
TLDL
For the very simple sequential read, FIO reports is much slower than the NVMe SSD sequentail read capability.
Main Text
Hello everyone,
I have been facing an issue while trying to achieve the ...
- 147
0
votes
0
answers
246
views
Calculating the CPU-utilization given a round-robin scheduler with 1 ms time quantum, CPU-bound and I/O-bound tasks, and a context-switch time penalty
Consider a system running ten I/O-bound tasks and one CPU-bound task. Assume that the I/O-bound
tasks issue an I/O operation once for every millisecond of CPU computing and that each I/O operation ...
- 101
1
vote
0
answers
26
views
Latency on pselect caused by other user space processes
I have the following pre-condition on Linux:
I'm using a process to read serial data using pselect on a serial device /dev/ttySX.
Data comes in at a stable frequency of 400 Hz.
To optimize latency of ...
- 11
0
votes
0
answers
246
views
How to use vdbench to do multiple operations concurrently on the 3 file system
I want to use following vdbench parameter files to do muliple operations concurrently on the 3 file system, however, the execute failed as in below.
Can some one help to check what is wrong in the ...
- 1
2
votes
1
answer
177
views
Softirq delayed because of the Heavy I/O Operation
I am trying to copy the contents of a 1 GB ISO file (stored in USB) to my internal storage. As part of a test when i am doing repetitive copy operation from USB to my hard drive continuously I observe ...
- 21
3
votes
1
answer
580
views
I/O errors, but after running badblocks everything works again : how is that possible?
TLDR;
HDD seemed damaged. Unable to format partition (mkfs.ext4 I/O errors), even with a newly created GPT table. SMART test shows some errors. I was about to throw the disk away. Before that, out of ...
- 1,805
1
vote
1
answer
283
views
Dmesg reports a lot of "Unrecovered read errors" and yet rsync copies just fine. Why?
I've been copying moderately valuable data (nice to have, not the end of the world if it is lost) from a bunch of old HDDs to my new NAS with more reliable storage.
During reading I get a bunch or ...
- 1,370
2
votes
1
answer
448
views
Force rsync to output messages line-by-line
(I'm not sure if this question should go on StackOverflow or here. But the actual problem is with rsync, so on balance I reckon it's a Linux question. If admins disagree, by all means please migrate ...
- 143
0
votes
0
answers
119
views
mc command hangs with --version parameter when writing to stdout but works when piping to cat?
I observed odd behaviour of a utility binary (it's actually the minio client mc in a recent-ish version) when running it from the command line.
If I just run it like ./mc --version, the command hangs ...
- 147
2
votes
1
answer
57
views
time difference of rm single tar file between large and small size
If I have a file data.tar and one is 1 MB in size versus one that is 10 GB in size versus one that is 10.0 TB in size (yes 10 terabytes is correct) why is there a time difference in doing a rm on ...
- 7,423
3
votes
1
answer
729
views
How to disable/enable bounce-buffers in IOMMU?
I am trying to figure out how to disable bounce buffers used in IOMMU when the hardware IOMMU is used.
To give more context, when IOMMU_DEFAULT_DMA_STRICT is set in the kernel it enables strict IOTLB ...
- 95
0
votes
0
answers
264
views
Parallelism and I/O in Linux
I'm a bit confused about how the Linux kernel handles parallelism during I/O operations (if it handles it at all).
I assume it can concurrently operate on file descriptors, but does it achieve ...
- 1
0
votes
0
answers
284
views
Oracle Linux 8.x - Performance Test (Stress / Load / IO / Network Usage / Memory Usage)
I want to do performance test on Oracle Linux 8 KVM guest server (It will be used as Application server (WLS 14c) and/or Oracle DB server).
What are the best tools I can provide this with? I saw an ...
1
vote
1
answer
926
views
How to monitor the I/O of a network-mounted disk in a process specific manner?
We have a disk mounted via network. nfsiostat revealed some basic stats for the whole disk:
$ sudo nfsiostat -d /mountpoint
10.20.xxx.xx:/source mounted on /mountpoint:
op/s rpc bklog
7.01
...
- 11
4
votes
1
answer
2k
views
How does mmio get routed to io devices?
I am trying to understand how IO devices are mapped into the 'regular' memory address space on modern x86 machines running Linux.
Some details which I am trying to make sense of are:
cat /proc/iomem ...
- 201
0
votes
1
answer
356
views
What makes sfill so slow?
sfill -Illv makes a file with data from /dev/urandom until it fills the partition.
Making a large file like this takes a lot of time. iotop tells me it is writing about 1 MB/s.
Though dd if=/dev/...
- 3
0
votes
1
answer
100
views
bash IO redirection, how du dup() STDERR?
Consider the following example:
exec 10>&2 # duplicate STDERR to fd 10
{ echo ok; inexistantcommand; } > /tmp/both 2>&1 10>/tmp/err
exec 10>&- # close fd 10
I expect /tmp/...
2
votes
1
answer
137
views
bash IO redirection with tee, without complicated syntax, how it works?
One way to do implicit dup(2) in bash with the output in fd {10,11,12} in a real use-case and close it right after, based on this:
$ cat tags
desktop-19.9.0
foobar-1.2.3
desktop-22.9.0
mobile-24.10.0
...
- 34.4k
1
vote
1
answer
278
views
Bash IO redirection, open & close 'fd', can someone explain?
I try to understand this concept for years, with at this time some incomplete knowledge of this.
Is anyone here can make a kind of a tutorial or good explanations with examples on how to
open a new ...
0
votes
0
answers
166
views
Gparted enable more logging
I cannot find a way for Gparted to produce more informative logs in the manual. What I am looking for is to find out why exactly did e cartain I/O operation fail (more details about the SCSI command ...
1
vote
1
answer
538
views
cgroup v2 io.max limit not applied
I can't seem to be able to apply io.max limit through cgroups v2 on Rocky Linux 9.
Setup:
# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
...
├─nvme0n1p5 259:5 0 444.9G 0 part
│ └─...
- 105
0
votes
0
answers
1k
views
How to get disk I/O activity by files and processes?
On Red Hat 8 I sometimes get huge disk I/O operations.
I can see disk I/O activities by using sudo iotop -oP command.
Output is:
PID | USER | DISK READ | DISK WRITE
---- ------ ----------- -----------
...
- 101
0
votes
1
answer
2k
views
Isolating I/O issue with NVME or hardware?
Hardware:
Samsung 980 PRO M.2 NVMe SSD (MZ-V8P2T0BW) (2TB)
Beelink GTR6, with the SSD in the NVMe slot
Since the hardware arrived, I've installed Ubuntu Server on it as well as a bunch of services (...
- 101
2
votes
0
answers
619
views
Is there a GUI linux application that shows per mounted disk i/o transfer rate in a graphic?
Is there a GUI linux application that shows per mounted disk i/o transfer rate in a graphic?
(Like gnome-system-monitor do for the swap mount but for each mounted drive)
I'm looking for an application ...
user330163
1
vote
0
answers
128
views
host hangs after cat /dev/null > bigfile.log
I found a big log file(2.7 TB) on my disk, so I decided to empty it with the following command:
cat /dev/null > bigfile.log
After I executed this cmd, I lost my ssh connection. When I logged in ...
- 11
2
votes
1
answer
47
views
input redirection to common commands like userdel [duplicate]
Is it possible to use users.txt a file that contains some users to redirect the input to
userdel
like this
userdel < users.txt
my question applpies to these common commands not only userdel.
1
vote
1
answer
1k
views
What is the impact of different Linux I/O APIs on performance benchmarking of NVMe SSDs? (libaio vs SPDK vs io_uring)
I am benchmarking NVMe SSDs on my Linux server, with the aim to achieve the IOPS, BW and Latency values as mentioned in the product specifications.
I'm using FIO as workload generator, and used libaio ...
1
vote
1
answer
223
views
How to write a one-page file to disk in one flush?
Say I have 4096 bytes of data that I want to save in a file. It should take exactly one page and can be flushed only once. However, if I do it using multiple calls to kernel i.e. write piece by piece, ...
- 214
0
votes
0
answers
186
views
in linux, system is slowed very much and CPU 100% when I copy from (non-system-based) usb HDD to another usbHDD. Why?
in linux, system is slowed very much and CPU 100% when I copy from (non-system-based) usb HDD to another usbHDD. Why?
My CPU is 100%. I was told that CPU can be 100% without system be slowed down....
- 242
0
votes
1
answer
333
views
Soft link will pass filename as 1st parameter to target?
https://github.com/adurbin/iotools
Original usage is:
./iotools rdmsr x xxxxx
when I doing:
ln -sf iotools rdmsr
then I can do:
rdmsr x xxxxx
Why??
because when doing soft linking, it will pass the ...
- 905
-1
votes
2
answers
60
views
When io util is relatively high, will Linux read and write hard disks return EIO?
When input/output util is relatively high, will Linux read and write hard disks return EIO?
- 1
1
vote
0
answers
126
views
why top command in linux shows high us and low wa while cp many large files simultaneously?
I am copying 10 large files(which may costs 1 hour) simultaneously, and I used top command to check the cpu usage, but I found out 100.0 us and 0.0 wa. shouldn't it be IO-bound instead of CPU-bound? I ...
- 11
0
votes
1
answer
132
views
Why top is not reporting high %wa on too many write sys calls
My HTTP application suddenly started taking extra time to start up. During this phase, it loads the data from the disk to the in-memory data structure.
I doubt it is due to the extensive logging ...
- 11