Questions tagged [io]

I/O is an acronym for input/output, which refers to the mechanism or process for the transfer of information between one system or component and another.

Filter by
Sorted by
Tagged with
0 votes
0 answers
25 views

commit argument in ext4 fs vs commit mount option in fstab

Target Improve the I/O performance of ext4 file system, specifically. Background I got to know about noatime and commit mount options for fstab, and also a warning that the former can be a breaking ...
user avatar
2 votes
1 answer
35 views

How to limit IO bandwidth using systemd?

I'm trying to limit IO usage of a service (or process/command). (Limit for IO read/write on hardware device, not for network.) I found IOAccounting from systemd: systemd-run --scope --user -p "...
user avatar
0 votes
0 answers
31 views

Is there any way to achieve prioritized IO operations in Linux 5.xx?

I run a lot of long-lived and io-heavy processes in addition to interactive processes on my machine. Unfortunately, they slow the interactive operations down to a crawl despite my best efforts. I ...
user avatar
  • 219
1 vote
1 answer
107 views

why is iotop showing "?err"

when I run iotop, the second column "PRIO" shows ?err everywhere. What does this mean? I did not find any mention of err in the manual page. I am using custom compiled kernel. Could it be ...
user avatar
0 votes
1 answer
46 views

dd: when is mandatory use bs and count together?

I am reading many tutorials about the dd command. There are some examples including the bs and count parameters. Some of them where each one is isolated over the other and other where both are used ...
user avatar
0 votes
0 answers
21 views

Are iodepth and iodepth_batch the same in fio?

Wanted to know the difference between iodepth parameter and iodepth_batch parameter in fio. Here is what is mentioned in fio read the docs : iodepth=int Number of I/O units to keep in flight against ...
user avatar
1 vote
0 answers
89 views

PCIE USB card not working. What's the missing "... resource 0 [io ...]" line in dmesg meaning?

Problem with 4x USB controller PCIE cards seems to be not correctly installed. Situation: We want to use USB Vision cameras with USB adapter cards, providing as much bandwidth as possible. So we tried ...
user avatar
  • 11
0 votes
0 answers
23 views

redirect TTY input to output without serial cable

I'm using simulation software for unix (ucsim) that runs two simulated programs that supposed to connect to each other via its own simulated serial ports. The software provides me with these methods ...
user avatar
  • 1
2 votes
1 answer
48 views

`rsync` deletes all after input/output error mapping

I'm running a rsync command between two servers with the following command: rsync -auv -P --delete --delete-excluded --stats --progress -e "ssh -i ~/.ssh/$ssh_pkey -p $port -l $user" $...
user avatar
1 vote
0 answers
130 views

UIO instance of a device-tree subnode

I have the following device-tree node which is auto generated by Xilinx PetaLinux in a proper dtsi file (which cannot be modified): axi_dma_0: dma@a0000000 { #dma-cells = <1>; ...
user avatar
  • 607
1 vote
0 answers
23 views

Were there any programs besides editors that relied on on the O_NDELAY file status flag (predecessor to O_NONBLOCK)

Recently, I've been researching on the O_NONBLOCK feature specified in the standard. From what I've seen so far, the predecessor of this flag - O_NDELAY had existed all the way back to UNIX System III....
user avatar
  • 429
0 votes
1 answer
106 views

How do I get the total read and write IOPS for each process or thread in Linux?

I want to find the process with the highest IOPS in the system. What should I do ?
user avatar
2 votes
2 answers
97 views

Linux Page Cache Performance versus memcpy

I am benchmarking the write performance in the case that Linux writes the data to the page cache without throttling the process or syncing the data to the disk. The simple experiment I am doing looks ...
user avatar
1 vote
1 answer
37 views

from where htop collects IO read/write data for one current user proccess without root?

I cant find a way to capture htop output for a single process about IO_RATE IO_READ_RATE IO_WRITE_RATE. I need it (the last few seconds IO (or just 1s), not the overall accumulated IO) to determine if ...
user avatar
2 votes
0 answers
93 views

When does Linux flush data to the disk?

I know that usually the answer to this question is "when it sees fit", or some other obscure answer. And many times I saw that if the host gets hard booted, some of files that got recently ...
user avatar
  • 1,480
0 votes
1 answer
27 views

Is there a Linux Kernel Interface to Notify Userspace Application about Disk I/O?

I have an embedded x86 computer board which does not have any kind of I/O activity LED (e.g. HDD LED). I am not using SATA drives so the normal workaround of using SATA PIN 11 as an activity indicator ...
user avatar
0 votes
2 answers
480 views

Files having Input/output error: how to completely ignore them as if they don't exist

Due to hardware problems, there are some Input/output errors. Some of my hard disk sectors are bad. # find . -name 'cpp-service.zip' find: ‘./.cache/chromium’: Input/output error find: ‘./.config/...
user avatar
  • 1,315
5 votes
1 answer
288 views

Difference between posix_fadvise and readahead

As per manpage, readahead is Linux specific: CONFORMING TO The readahead() system call is Linux-specific, and its use should be avoided in portable applications. and posix_fadvise is portable: ...
user avatar
0 votes
0 answers
9 views

what side effects should I expect from setting all jbd2 processes ionice to idle?

I know whatever it is doing will be slower but what exactly? I mean also what practical side effects can I expect, in a sense that, should I avoid idle it for any reason? Is it safe? here: anPid=(`...
user avatar
0 votes
1 answer
63 views

Pipe output of two commands into another script as arguments

ntopng can handle multiple network interfaces and I've a script that takes the name of the interface and outputs it in to stdout. I can pipe this output into ntopng my_packet_tracer.sh | ntopng -i - ...
user avatar
2 votes
2 answers
101 views

How to tell Debian to buffer USB stick writes more aggressively?

I'm trying to find a way to speed up my IO to/from a USB stick on my Debian system, maybe with some caching. Background: I've written a script that does debootstrap, i.e. installing a Debian system to ...
user avatar
4 votes
1 answer
164 views

Utility to show disk IO read/write summary for a task/command/program

I'm pretty sure the Linux kernel has a feature which allows to track all the reads and writes (IO) of an application and all its children however I haven't seen any utilities which can calculate it ...
user avatar
0 votes
0 answers
13 views

Is there a command-line option to get IO information per disk when using pcp-dstat?

I need to gather IO information per disk using pcp-dstat. However, while dstat (its predecessor) directly displays IO activity per disk, pcp-dstat only gives IO information for the system overall. ...
user avatar
2 votes
1 answer
66 views

hard drive read throughput terrible after running badblocks, recoverable?

I have a collection of new drives, a friend recommended I run badblocks on the new drives before using them. I ran badblocks on some of the new drives before imaging them badblocks -v -s -w -b 4096 -...
user avatar
  • 3,844
0 votes
0 answers
32 views

qemu-img making the server inaccessible (High iowait)

I have a dedicated server with 2 * 1TB NVMe Disk drives, centos 7 os. With virtualizor installed. When i run qemu-img convert : qemu-img convert -f raw -O raw /var/virtualizor/kvm/windows.img /vgs/...
user avatar
  • 3
4 votes
0 answers
188 views

/proc/diskstats disk time spent writing more than 1000ms*CPU-core-count per second

I'm interested in how could one interpret the results of procfs-diskstats. It looks like an API, but I see no exhaustive explanation in Internet or man pages or Linux kernel Documentation. There is a ...
user avatar
  • 265
1 vote
2 answers
52 views

Is there a tool that can read and write to disk at constant throughput?

I'm looking for a tool that can read and write data at a constant target throughput, say 3Mb/s, rather than pushing the I/O system to its limits. I then intend to monitor various metrics whilst this ...
user avatar
2 votes
0 answers
179 views

IO Throttling After Reaching dirty_background_ratio

I am trying to understand the Linux IO throttling mechanism. This is what I got so far: By calling the write (fd, data, size) system call, the data will be written into the OS cache as dirty pages. ...
user avatar
1 vote
1 answer
143 views

How is /dev/null implemented?

I don't understand what happens when a process writes to this special file. What happens to all the bytes? And why is writing to /dev/null not cpu-intensive? If you can answer in terms of references ...
user avatar
1 vote
2 answers
265 views

How to overwrite a file in the shell using < and > for the same file? [duplicate]

When I do my_script < filename.txt > filename.txt, the file is overwritten and truncated. Is there some way on the Unix command line to specify that redirection is not done concurrently, i.e. ...
user avatar
1 vote
1 answer
58 views

Occasional I/O Errors when attempting to transfer some files but others work fine

I'm running on Red Hat Enterprise Linux and when I attempt to transfer files through the file browser from one machine to a new directory on a server, I run into an issue where some files are able to ...
user avatar
  • 11
1 vote
1 answer
1k views

update-initramfs: failed for /boot/initrd.img-4.15.0-112-generic with 1

descriptions: when i run dist-upgrade i ran into the problem with below detail 1 not fully installed or removed. After this operation, 0 B of additional disk space will be used. perl: warning: Setting ...
user avatar
0 votes
0 answers
52 views

How to show the output for the execution for any command and/or script in stdout and file but keeping the color in the stdout?

For some commands, in the terminal they print with some colors in the stdout, for example: git status mvn help:help -Ddetail=true gradle build Any Linux command (ls [-...], etc) Note: it applies for ...
user avatar
0 votes
0 answers
61 views

Server is slow, there's no workload, no apparent reason, only shutdown solves the problem

I manage a server that is now approximately 5 years old. It has Ubuntu Server installed. There had been a handful of times where suddenly it became super slow to the point it was basically unusable. ...
user avatar
2 votes
0 answers
22 views

Disabling user I/O during loop (temporarily)

I have a shell script that backs up my git repos that are all saved in one specific file. I wanted to add a cool "spinning load thingy" just for fun and that's pretty simple. The problem is ...
user avatar
0 votes
0 answers
47 views

How can I monitor whether disk activity is sychronous or asynchronous?

My Google-fu simply cannot find an answer to this. If I have a process with heavy I/O activity, how can I check whether it's using asynchronous or synchronous writes? (I want this information to ...
user avatar
1 vote
0 answers
58 views

NVME slower on host than in VM

I first noticed some strange performance hits when I switched from Vagrant to Docker for my local development setup. After investigating why my databases were slower in this new environment, I came to ...
user avatar
  • 111
1 vote
0 answers
220 views

Why is SSHFS so much less stable than SSH?

If I'm on a very slow and/or slightly hiccupy internet connection, and I ssh foo@bar.baz, the connection will mostly stay up. If there's particularly bad lag, output (and echoed input) will take ...
user avatar
0 votes
0 answers
47 views

Could it be possible to throttle IOs using LD_PRELOAD?

I have a job on a managed Kubernetes cluster that is a bit IO intensive. Nothing crazy, it downloads about 1gb of data, writes it on /tmp folders, does some minor edits, compress and encrypts it, and ...
user avatar
0 votes
2 answers
494 views

Is It OK to use oflag=direct and conv=fsync together in dd?

My target device is a hard drive with 4096 bs. I want to bypass caches and write to storage directly or as soon as possible. Speed is not my primary concern: 30 hours is too much, but the difference ...
user avatar
  • 267
1 vote
1 answer
50 views

bash I/O redirection - how to append to stderr

I have a script that loops over some big collection of data and performs some lenghty operations. Then i need to sort | uniq -c its output. So to let it know that its alive, I print a dot every N ...
user avatar
  • 153
1 vote
1 answer
135 views

Unclear htop output when measuring IO usage of a dummy process

I write a simple test.sh script which only does this: while true; do cp ./target-file ./target-file.tmp; done wherein target-file is created using dd if=/dev/urandom of=target-file bs=1M count=100 and ...
user avatar
1 vote
1 answer
127 views

Linux reads ahead when concurrent reads done on same file

Linux performs readaheads (specified in /sys/block//queue/read_ahead_kb) when a file is read sequentially. Interested OS : Red Hat Linux Interested File System : xfs, ext4 What is the criteria for ...
user avatar
  • 111
0 votes
1 answer
46 views

How does linux redirection manage fast input and slow output?

When I do this, $ cat ssd/file1 > hdd/file1 and read from ssd is faster than writing in hdd, what happens to the data that cat is reading? Is there any communication between input and output, which ...
user avatar
0 votes
1 answer
46 views

Sysdig does not show exit event for write syscall

I'm trying to capture write to stdout (or stderr), but apparently the actual data is related to the exit event. I wrote a simple C program that writes to stdout and stderr. #include <stdio.h> ...
user avatar
1 vote
0 answers
192 views

System keeps trying to access/read my laptop's empty CD drive

I'm using a 16-to-18-years-old laptop, a Clevo Zeus M3CW, on which I recently installed antiX Linux 19.3. The kernel version is 4.9.235 (compiled for 486). Now, whenever I boot, the kernel complains ...
user avatar
  • 7,249
1 vote
0 answers
32 views

subshell and Input/Output Redirections questions

I have a couple of questions about the behavior of the shell i/o echo apple | echo banana The output here is only banana But in this command: echo apple | { echo banana; tail; } the output is ...
user avatar
  • 153
0 votes
0 answers
256 views

Why I don't see any increase in "Dirty" memory when writing to a file?

I read a bunch of stuff about "Write-back" mechanism today and wanted to try an experiment. EDIT: file downloaded from the Internet shows up in Dirty stats Based on the comments, I tried to ...
user avatar
0 votes
0 answers
32 views

Modify the generic block layer in a Linux system

For an university project, I need to modify a kernel layer in a Linux operating system. After modification I should prove performance improvements. Concrete I should modify the generic block layer and ...
user avatar
  • 101
0 votes
1 answer
105 views

dd bs=X count=1 reads less than X bytes [duplicate]

I have dd from GNU coreutils 8.32. When I run { echo a; sleep 1; echo b; } | dd bs=4 count=1 then I get a 0+1 records in 0+1 records out 2 bytes copied, 2.0381e-05 s, 98.1 kB/s dd terminates during ...
user avatar
  • 551

1
2 3 4 5
12