Questions tagged [io]

I/O is an acronym for input/output, which refers to the mechanism or process for the transfer of information between one system or component and another.

Filter by
Sorted by
Tagged with
0
votes
0answers
10 views

Is it more efficient to use file IO or swap space for large temporary storage?

(Cross-posted from SO.) I have a program that needs to produce a large quantity of temporary data (in the 10-100 gigabyte range) that I don't expect to fit in memory. The data is only used internally ...
0
votes
1answer
41 views

The difference between the udev rules and the proc sysfs for configuring the I/O scheduler? [closed]

Under which circumstances are the following config methods appropriate for use? "indirectly" use the udevadm via /etc/udev/rules.d/<iosched_file>: e.g. ACTION=="add|change", ...
1
vote
0answers
28 views

May a character device only handle blocking I/O?

Let's consider the following (imaginary) device: a clock which takes 1 second to query, then returns the current time. We want to write a character device driver for it, which supports read operations ...
0
votes
0answers
8 views

Some doubts of the I/O scheduler configuration with the partitioned sub-disks

Should I configure the same I/O scheduler with each partitioned sub-disks ? Could I configure the different I/O scheduler with each partitioned sub-disks ? # # # cat /sys/block/sdb/queue/...
1
vote
0answers
13 views

Writing to a device hangs, how can I troubleshoot this?

I have a teensy 3.6 as /dev/ttyACM0 I noticed that when I used serial.write() in python it would hang. I opened the Arduino IDE serial monitor and I can see the output. When I write in the serial ...
0
votes
0answers
10 views

Debugging slow disk speed: Why discrepancies between dd, raw disk, file systems, iozone?

I got a new 8TB Seagate disk which I connect via USB 3 and use with ZFS. When I am copying files via Samba (e.g., large 4GB file) I barely get morethan 4MB/s. When I am directly writing to the disk (e....
1
vote
1answer
25 views

Stream changes in available memory

I've come up with the following command to check changes in memory use: free -s 1 -h | awk 'NR%4==2' This shows output like this: Mem: 125G 32G 82G 404M 10G ...
-4
votes
1answer
69 views

Why performance degradation if read/write operations are not performed in multiples of the disk’s block size?

In Love's Linux System Programming, in CHAPTER 3 Buffered I/O Recall from Chapter 1 that the block is an abstraction representing the smallest unit of storage on a filesystem. Inside the kernel, all ...
0
votes
1answer
28 views

What value of “wa” from the top command is too high?

I have a server with 64 GB of ram, 16 CPU cores and an SSD drive. I am running a relational database on the machine. I would like to increase the writes to the database, and my guess would be that the ...
0
votes
0answers
48 views

Can a file descriptor ready for IO become unready before any IO operation is performed on it?

In The Linux Programming Interface, on p1327 in 63.1 Overview of 63 Alternative Models In effect, I/O multiplexing, signal-driven I/O, and epoll are all methods of achiev- ing the same result—...
0
votes
1answer
26 views

What is the path of data transfer when using memory mapped file?

Is there any difference between the data transfer paths using read()/write() and using mmap() on a file? What does "kernel" mean in https://stackoverflow.com/a/41419353? mmap doesn't ...
0
votes
1answer
37 views

How does preemption work during hardware I/O?

Say I have a device that uses SPI or another communication protocol to talk with my computer running Linux, Raspberry Pi for instance. For this communication to succeed, no interruptions are allowed, ...
0
votes
2answers
23 views

Get output and input into daemon processes

Background When a SIGHUP is sent to a running nginx master process to reconfigure, if there're password encrypted certificate key, nginx(specifically the openssl library) will ask for input, as in the ...
1
vote
1answer
53 views

Difference between BUFSIZ from stdio.h and block size from blockdev?

I'm reading The Unix Programming Environment (1984). In Chapter 7 (Unix System Calls) the authors note: While the number of bytes to be read or written is not restricted, the most cmmmon values are 1,...
0
votes
1answer
74 views

dm_crypt / kworker hogging IO and causing system freeze

I'm not sure how to debug this, but I've noticed if I am performing a task that requires a large amount of disk reads/write (such as updating a large postgres table) that periodically actual reads and ...
2
votes
1answer
58 views

Why does io_uring have a layer of indirection for the submission queue?

From the paper on io_uring, the submission ring contains indices into a submission array, where the submission event itself is stored. The documentation explains this layer of indirection as follows: ...
0
votes
1answer
26 views

Unclosed IO resources disrupt input at next shell prompt

Sometimes when I run a terminal program that spawns a child process, (e.g. IO.popen in a Ruby script), if the IO handle for the subprocess is not properly cleaned up (e.g. IO.close), then when the ...
1
vote
0answers
43 views

NGINX directio is not working (O_DIRECT flag in open(2) sys call )

Nginx configuration is location /conent/ { directio 200k; directio_alignment 4k; } Using SSD with ext4 file system with 4k block size, and journaling has been disabled, linux kernal version is 5.4., ...
0
votes
0answers
16 views

encountered HDD I/O error?

I'm not sure if this is the proper place to ask about the solution to this issue. I've been slowly piecing together some fragments online, but wanted to ask here. Essentially I think there is some ...
5
votes
2answers
165 views

Why are processes blocked by I/O in case of heavy system load?

I have a workstation(2x Intel Xeon family CPUs and 128GiB of RAM) running several virtual machines and while the combined CPU usage is <30%, then the load average is between 20 and 25. For example, ...
1
vote
1answer
40 views

diagnose I/O error on WD Blue SSD

I have a 1TB WD Blue SSD. It has two partitions, Disk /dev/sdd: 931.53 GiB, 1000204886016 bytes, 1953525168 sectors Disk model: 2115 Units: sectors of 1 * 512 = 512 bytes Sector size (...
0
votes
0answers
23 views

Using “expect” command to run an interactive program fails

I am trying to solve my question that I've asked here using expect script instead of file redirection. Unfortunately, this solution doesn't work as well (the same behaviour as described in the ...
3
votes
2answers
308 views

Feeding a command using file redirection or pipe doesn't always work

I've downloaded this program construct2d and compiled it using GNU Fortran gfortran 9.3.0. You can compile the program using gnu make: make (compilation time: 10 seconds on my PC running Ubuntu 20.04 ...
0
votes
0answers
13 views

Test case for Deadline IO Scheduler

I am trying to measure the impact of Deadline IO scheduler on the performance of Block IO. My setup includes: Kernel Version: 4.4.154 Storage device: eMMC 64GB Distribution: VERSION="18.04.4 LTS ...
1
vote
2answers
45 views

High IO rate when the system out of RAM

I found that when the system goes out of memory, the disk IO usage will be very high. It seems many processes are reading from hard disk frantically (check htop output below). When I kill a process ...
1
vote
0answers
323 views

How can one use Nvidia RTX IO on Linux to allow the GPU to directly access the storage (SSD) with only a slight CPU overhead?

I saw in the 2020-09-01 Official Launch Event for the NVIDIA GeForce RTX 30 Series that the RTX IO in the Nvidia GeForce RTX 30 Series allows the GPU to directly access the storage (SSD) with only a ...
2
votes
1answer
49 views

What is the meaning of synchronous and asynchronous requests in Linux Kyber I/O Scheduler?

While studying I/O Schedulers, I've read some documents for I/O Schedulers in Linux Kernel. Especially for Kyber Scheduler, there are some confusing things. In the articles in following link ( https://...
0
votes
1answer
55 views

Input Output errors on disk drives

I am completely new to the ins and outs of linux. I am trying to check my drives by running an exerciser software, blast, that checks the read and write capabilities. I run write verify loop on the ...
1
vote
1answer
32 views

How can I see which process does the most writes on a specific partition?

I am using Debian Buster and would like to find out which process does the most writes on a specific partition, just like iotop but limited to a single block device?
0
votes
1answer
70 views

Why htop shows no use of IO when iotop does?

I just started to pvmove --alloc anywhere on an luks2 encrypted PV. The commands takes a long time, and as expected, in iotop I can see the commands invoked for this operation are taking a fair bit of ...
3
votes
1answer
132 views

sha1sum reports that checksums do not match even though they are identical

Perhaps I'm doing something very stupid here, but Google hasn't been very helpful in resolving the problem. I have an archive of files I made for backup purposes. I've generated an SHA1 checksum file ...
0
votes
0answers
14 views

Is there a way to track the volume of a disk traffic from/to a file

I'm running a VM (qemu) that has its virtual disk on my hard drive. It works pretty slow so I'm thinking of moving that virtual disk to SSD. The only thing I am concerned about is whether it would be ...
0
votes
1answer
41 views

How to execute bash commands form one bash to another bash process?

I have two bash processes running. Let's name them bash_1 and bash_2. I wish to execute command in bash_1 from bash_2. I am following this method to achieve this. Get the dev file for IO in bash_1 ...
4
votes
1answer
215 views

dd “direct” versus “nocache”

According to the manual for dd, there are two options that can cause I/O to bypass the filesystem cache: direct and nocache. When using these flags to, for example, copy files or copy raw data to/from ...
1
vote
2answers
365 views

How to change the IO policy to noop on centos 8 for all kernel?

I want to change the IO policy to noop for all the VMs of all kernel version. (Even if there is a future updated version also) cat /sys/block/sda/queue/scheduler Output as follow [mq-deadline] kyber ...
0
votes
1answer
24 views

Root partition failing, usb, ext4, Archlinux

Problem: Installed Archlinux on a USB pen-drive (ext4) about a month ago. Everything was working fine up until recently. After a while I would not be able to execute commands (except bash built-ins). $...
1
vote
2answers
38 views

How to clear the content of a file which is locked by a running process?

I am running JMeter on an Ubuntu machine. And running a 12 hour test.  During this I am observing the log file is increasing to a very big size (around 18 GB). Is there any way I can use inside a cron ...
0
votes
0answers
35 views

How to mount a Windows SD card? (Errors on dmesg)

So I have a sd card that I use for my DSLR Camera, and I believe the camera only has compatibility for windows file systems (I believe it's FAT32?). I found this out one day by my camera not working ...
0
votes
0answers
51 views

With io_uring is it possible to do an atomic write and tell?

I would like to append to a file opened in O_APPEND and know where the data was written. Writing and then making a tell call isn't useful as between the system calls there could be a context switch ...
0
votes
0answers
25 views

How to keep system responsive during high io load?

System is Raspberry Pi 3. OS is Raspbian 10. Something as simple as running dd will cause the entire system to lock until it is finished. I have read this: https://unix.stackexchange.com/a/5639/...
0
votes
0answers
46 views

Effect of Buffer Size in File I/O in Unix

I am trying to understand the inner workings of Unix based OSs. I was reading on buffered I/O, and how the buffer size affects the number of system calls made, which in turn affects the total time ...
1
vote
1answer
95 views

Identify if IO bottleneck is read or write

I use ddrescue to image failing disk to sparse files - often the files become highly fragmented (>50k fragments). I suspect that sometimes the imaging speed degrades because of the fragmentation. Is ...
0
votes
1answer
47 views

How to force kill program stuck while accessing IO (firefox)

I use firefox to display some files over an sshfs mountpoint... However, it happens that this mountpoint gets disconnected; if I inadvertantly reload a page, firefox will be forever stuck and cannot ...
1
vote
2answers
126 views

Why is overwiting a file slower than writing to a new file?

I noticed that sequentially overwriting an existing file may be much slower than writing a new file. Example: mkdir del cd del fio --name=seqwrite --rw=write --ioengine=sync --bs=128KiB --size=10GiB ...
2
votes
2answers
2k views

Why is dd so slow on a NVMe SSD?

I recently watched this video and in it, the author explains that a ram disk is far faster than HDD/SSD when using dd command. I understand why this is the case. What I don't understand, however, is ...
3
votes
1answer
195 views

crontab command fails to create temporary directory

On two of my VMs the crontab -e command fails to create a directory in /tmp but it works with mkdir: /tmp/crontab.oxyaBy: Input/output error Creation of temporary crontab file failed - aborting ...
0
votes
0answers
19 views

Replace root folder of writing paths used in process

I have a process that creates and uses temporary files. When I run many instances of this process the disk throughput becomes a bottleneck. I know that I can use a temporary file system mounted in RAM,...
1
vote
1answer
884 views

What's the best dirty_background_ratio and dirty_ratio for my usage?

So I'm playing with dirty_background_ratio and dirty_ratio and hoping to find the right parameters with your professional help. For now I'm using: vm.dirty_background_ratio = 20 vm.dirty_ratio = 60 ...
2
votes
1answer
174 views

How well do concurrent writes scale with io_uring?

Background: I'm very intrigued by the potential performance gains of io_uring, but I don't have enough familiarity with C, the Linux kernel, or the inner workings of block devices to implement ...
1
vote
1answer
192 views

“-bash: fork: Cannot allocate memory” at every command I enter, I'm stuck

I executed a python script that uses basically all the memory of my vps because it does I/O operations with files of huge dimensions. I executed this script as root. I tried to do a killall python3 ...

1
2 3 4 5
11