Questions tagged [io]

I/O is an acronym for input/output, which refers to the mechanism or process for the transfer of information between one system or component and another.

Filter by
Sorted by
Tagged with
2
votes
1answer
15 views

Does it have 3 levels of buffering, or 2 levels of buffering, when using stdio library?

From The Linux Programming Interface: (I've read the related sections. I pasted this sum-up diagram just for illustration.) I know that for direct IO which uses read and write system calls. There is ...
0
votes
0answers
25 views

renice and ionice processes that are hogging resources

I've noticed that Linux grids to a halt as soon as 1 application or 2 take up all the CPU or Disk IO resources. The best solution I've found so far is to renice the processes to idle, this is of ...
0
votes
1answer
19 views

How to evaluate SAR data?

I know how to collect and make use of the data gathered by the sar tool, but I don't know how to interpret it correctly. For example, PAGING: I can see that my pgpin is 300.000 but how can I tell if ...
0
votes
0answers
27 views

Is there a way to find out which process is writing to a specific disk

Right now I can use dstat to find out the I/O status of individual disks, and iotop to find out the I/O status of individual processes, but I can't find a tool that can combine these two functions. I ...
-1
votes
2answers
82 views

Downloading and writing a text file onto tmpfs is slower than writing to disk, why?

I have a tmpfs named /rtmp/ that has 1GB of RAM allocated to it on Ubuntu OS. Using a bash script, I am testing to see if it is faster using to write a small text file to the hard disk or if it is ...
2
votes
4answers
222 views

`wc -l` alternative that displays the count, as it receives the lines

I'm using wc -l to count the lines in output of a command, as the input is piped to it. commad | wc -l This works fine, but if command is doing some heavy computation, this is quite slow. Is there ...
0
votes
0answers
11 views

How can I delay/queue all disk activity on a certain directory/disk until a certain point?

I would like to be able to issue a command in a script that would stop disk activity to a certain directory (or disk) and queue it until I issue another command that would 'resume' the disk and all ...
0
votes
1answer
38 views

linux stress, impose work on specific disks

Have a server running centos 7.6, and it has 4 ssd's as Raid-0 mounted as /scratch/ I have the linux program stress-1.0.4-16 and I just learned stress-ng existed. Is there a way with stress to tell ...
0
votes
1answer
165 views

How do I investigate or fix random IO HDD & system freezes on linux?

My home desktop system is Ubuntu 18.04.1 with kernel regularly updated, currently 5.3.0. From time to time, mostly when browsing but not necessarily, the system becomes slow on IO: - hdd LED ...
0
votes
0answers
208 views

How to figure out cause of high kernel/kworker CPU usage?

Sometimes the kernel uses up 40-50% of my CPU, causing lag in other programs. At the times this happens, the output of iotop looks like this: Total DISK READ : 2.96 K/s | Total DISK WRITE : ...
0
votes
0answers
14 views

How to calculate I/O of a virtualization platform?

We use oVirt as our virtualization platform, with 8 hypervisors. We use an MSA1040 as our main storage with iSCSI. We're hitting some I/O problems, and we're searching for a tool to monitor and ...
0
votes
1answer
19 views

Is it possible to find amount of data that come and go through HBA ports?

I have a ProLiant server with a HBA card (2 ports). $ ls /sys/class/fc_host host2 host3 and also I have a 20T SAN storage that is connected to San switch via those ports. Is it possible to ...
0
votes
0answers
54 views

How does history print to the terminal?

EDIT: As per @JdeBP's comment, I've learned that the history command and the history recall functionality of interactive terminals are two separate things. I am, however, still curious about how the ...
0
votes
0answers
47 views

How to fully enable io.latency controller?

I'm trying to setup my Kodi to never skip. Currently I focus on providing absolute priority to Kodi's I/O. System is Fedora 31. Kernel 5.4.0, has CONFIG_BLK_CGROUP_IOLATENCY=y I've started with ...
3
votes
1answer
52 views

Linux aggressively swaps when doing large IO operations

I have a strange problem on my Debian 10 desktop system when I am doing large IO operations like copy large files from/to external drivers or also on the internal disc. I could pin this down to the ...
4
votes
1answer
179 views

Why does I/O cause my system to become almost nonresponsive, and how to fix it?

I'm recovering data from a hard disk. The disk is in good working order; a collection of files were deleted by accident. I'm using photorec, with the location to store recovered files on another ...
0
votes
0answers
29 views

Wanted to check Disk IO Per process/services - per-proc IOP's (no of reads, no of writes), rw wait time, disk busy ness, inodes

I am looking for DISK IO per process/services in the linux machine : need following details: per-proc IOP's (no of reads, no of writes) rw wait time disk busy ness inodes I do not ...
0
votes
0answers
21 views

cd into directory with git lfs files takes very long

I have a (data) repository with ~50GB of data, split into <10 subdirs with files of 100 to 1000MB each. The repository is saved on a SSD, the workstation has 16 CPU Cores and 64GB of RAM. All ...
0
votes
0answers
12 views

zfs/linux - how to measure sync vs async I/O usage/calls by applications?

Is there a way to measure the amount of I/O issued by a process (or set of processes) that is sync vs async in nature? Assume that I have root access to the system (linux) but that I do not know in ...
1
vote
2answers
116 views

Run a process in background with std input and std output redirection

Is there a way to run a process in the background with both the input and the output redirected? I have a java program that is using the Java scanner with the System.in attached to it for input. So I ...
1
vote
1answer
99 views

Two (different) BW numbers in fio output!

fio output shows two bandwidth numbers at two places (for both read and write). What does these two numbers indicate? Which one should be considered for throughput test and for what the other one ...
0
votes
0answers
29 views

What Exactly is the Difference among Direct, Buffered, Synchronous, and Asynchronous I/O?

I have studied lots of documents but I got a little bit confused. Can you please explain the difference or the relation between the direct I/O, buffered/non-buffered I/O, asynchronous and synchronous ...
-1
votes
1answer
25 views

RAM bandwidth activity

I compile sources in /tmp mounted into RAM. I'd like to know fast it is! That's show/monitor I/O activity (MB/s) for this device. If I want to monitor HD bandwidth I would have used tools like iotop ...
2
votes
1answer
46 views

Is there a way to set pulseaudio to change audio output depending on the monitor that's playing audio has moved?

...I'm just using arcolinux (KDE5, 4.19-LT) so hopefully any generic solution will work without having to fight with the way the distro has set up the OS... ...The idea is to I suppose have something ...
2
votes
2answers
310 views

How to prevent disk i/o timeouts which cause disks to disconnect and data corruption with SMR disks?

I have these seagate disks (ST5000LM000 - note they are SMR) that when I put them under heavy write workloads the i/o utilization goes to 100% and throughput basically goes to zero. The disks are ...
2
votes
2answers
85 views

How does linux on x86_64 recover the 'lost' physical memory masked by memory-mapped IO?

/proc/iomem indicates that significant address space is mapped to PCI devices, such as a video card on my box: e0000000-efffffff : 0000:01:00.0 which is 250MB if my math is correct. On a 64 bit ...
0
votes
0answers
85 views

How does x86 northbridge/memory controller routes addresses for memory mapped io?

My current understanding is that northbridge/memory controller routes address accesses, based on some programmable rule, such that accesses to memory mapped region will be re-directed to southbridge ...
0
votes
0answers
13 views

How do I find where my process is writing temporary data to disk?

I am trying to figure out where my browser (Brave) writes its temporary data to disk. I'm on Ubuntu and I'm familiar with only gnome-system-monitor and ps. I know that SysInternals for Windows shows ...
1
vote
1answer
37 views

Python rare IOError for mounted samba drive

Procedure: Mount samba drive (sudo mount.cifs ......) Start jupyter notebook Run a script that opens a bunch of matlab files, and does some heavy lifting on them afterwards Leave the code run, come ...
1
vote
0answers
188 views

Disk IO bottleneck on Linux server

I've been facing Disk IO bottleneck on one of my streaming server, but couldn't identify the limiting factor. The issue might be on RAID card or in SSD due to which streaming application goes for the ...
-1
votes
1answer
557 views

how to run fio verify reliably in linux

I am using fio over disks exposed through iscsi. I am giving these params in fio fio --name=randwrite --ioengine=libaio --iodepth=64 --rw=randrw --rwmixread=50 --bs=4k-2M --direct=1 -filename=data --...
2
votes
0answers
27 views

How does a block device driver handle IO errors

When a block device gets a request to read a range of sectors, and one of those sectors has a read failure, how does the driver signal to the kernel that a partial read is available, and that the ...
1
vote
1answer
222 views

Fio Replay of Blktrace File Fails due to No Space Left on the Device

I am using fio-3.14 to replay a trace file created by blktrace. My linux version is Mint 18.3 with the kernel 4.10.0-38-generic. As a root, I followed the steps as below: First Step I applied an fio ...
1
vote
0answers
22 views

Why doesn't sync guarantee that buffers are flushed?

As far as I can see, the only use of sync is to ensure that there's no incomplete information, which is supposed to be safely written on disk but may actually be still only in a volatile buffer. ...
0
votes
1answer
32 views

An IO write operation can outlive a process?

To the best of my knowledge, when a process writes to a file it starts a system call. Among the required information, it expects a pointer to a buffer in the user space, filled with the data to write. ...
0
votes
1answer
228 views

Why does the new Linux “pressure stall information” for IO not show as 100%?

I noticed a shiny new line in atop (on my Fedora 30 install). man atop PSI - Pressure Stall Information. This line contains three percentages per category: average pressure percentage ...
2
votes
1answer
180 views

Could AIO fsync improve dpkg performance?

Could dpkg, the Debian package manager, gain a noticeable performance improvement by using one of the AIO fsync() operations, instead of sync_file_range() + fsync()? The [proposed] fsync2() API is ...
1
vote
0answers
52 views

cgroup IO control v.s. (the new types of) Linux AIO

In Linux v5.0, do some types of Asynchronous IO bypass the cgroup IO controllers? EDIT: buffered write() and fsync() might be relatively complex to analyse. I found an interesting post about ...
0
votes
1answer
59 views

Is there a kind of '/dev/null' to fake output file path?

Let says that I have a CLI which transforms a file_a into a file_b like so: transform --input /path/to/file_a --ouptut /path/to/file_b I have developed a program that use this CLI and need to unit ...
33
votes
5answers
4k views

How to redirect stdout to a file, and stdout+stderr to another one?

How can I achieve cmd >> file1 2>&1 1>>file2 That is, the stdout and stderr should redirect to one file (file1) and only stdout (file2) should redirect to another (both in append ...
2
votes
2answers
741 views

NVMe disk shows 80% io utilization, partitions show 0% io utilization

I have a CentOS 7 server (kernel 3.10.0-957.12.1.el7.x86_64) with 2 NVMe disks with the following setup: # lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT nvme0n1 259:0 0 477G 0 ...
5
votes
2answers
251 views

Why does hexdump try to read through EOF?

When run from a terminal, hexdump does not react to a single ^D at the beginning of the line, as cat, od, bc, etc. do, unless there was no input yet: prompt% hexdump -C <control-D> prompt% ...
1
vote
1answer
254 views

Understanding File Streams and File Descriptors

For extra context, I asked this question earlier, and thought I understood things, but not anymore. I know that echo ignores stdin. I know that stdin, stderr and stdout exist and, in this context, I ...
0
votes
0answers
196 views

ZFS Disk Image Performance

I am trying to move from LVM to ZFS as a backing store for raw virtual machine / disk images on a new server. I am using Ubuntu 18.04 and the SSDs are all Intel D3-S4510. The problem I am ...
0
votes
0answers
278 views

How to find max queue length for device?

Recently I ran into some I/O overload problrms on some of our machines and I have been searching to find thresholds for some metrics like: avgqu-sz await svctm Then I saw this post on Stack ...
0
votes
0answers
249 views

Who is causing extremely high CPU usage when writing to SSD?

When writing to SSD (regardless if over file system or directly into /dev/sda) I observe extremely high CPU usage, with top showing CPU: 0% usr 53% sys 0% nic 9% idle 27% io 0% irq 9% ...
0
votes
0answers
104 views

How to create a new character device under /proc/device file?

I am trying to create a node with mknod /tmp/ttyACM0 c 166 0 command but then No such device or address error returns when I try to open serial port with ttyACM0 because there is no device in proc/...
0
votes
0answers
40 views

How shell delivers user's input to program and shows program's output? [duplicate]

Since shell (for example, Bash) is just another program with its own stdin/stdout/stderr streams, how does it deliver user's input to running program? Here's simple program, that reads value from ...
1
vote
1answer
122 views

Enabling Asynchronous IO at HP-UX Operating System

We are using HP-UX Itanium version B.11.31 and this server is used for running Sybase database version ASE15.7. I would like to know how to enable Asynchronous IO at Operating System level. I went ...
0
votes
2answers
83 views

Can one processes have a descriptor that is non-blocking while another process have a descriptor referencing the same file/socket that is blocking?

I am more interested in sockets than regular files, but basically I want to know whether one process can "see" a socket as blocking where another process can see it as non-blocking. I am guessing yes,...

1
2 3 4 5
10