Questions tagged [io]

I/O is an acronym for input/output, which refers to the mechanism or process for the transfer of information between one system or component and another.

Filter by
Sorted by
Tagged with
0
votes
0answers
16 views

qemu-img making the server inaccessible (High iowait)

I have a dedicated server with 2 * 1TB NVMe Disk drives, centos 7 os. With virtualizor installed. When i run qemu-img convert : qemu-img convert -f raw -O raw /var/virtualizor/kvm/windows.img /vgs/...
4
votes
0answers
67 views

/proc/diskstats disk time spent writing more than 1000ms*CPU-core-count per second

I'm interested in how could one interpret the results of procfs-diskstats. It looks like an API, but I see no exhaustive explanation in Internet or man pages or Linux kernel Documentation. There is a ...
1
vote
2answers
47 views

Is there a tool that can read and write to disk at constant throughput?

I'm looking for a tool that can read and write data at a constant target throughput, say 3Mb/s, rather than pushing the I/O system to its limits. I then intend to monitor various metrics whilst this ...
2
votes
0answers
63 views

IO Throttling After Reaching dirty_background_ratio

I am trying to understand the Linux IO throttling mechanism. This is what I got so far: By calling the write (fd, data, size) system call, the data will be written into the OS cache as dirty pages. ...
0
votes
1answer
59 views

How is /dev/null implemented?

I don't understand what happens when a process writes to this special file. What happens to all the bytes? And why is writing to /dev/null not cpu-intensive? If you can answer in terms of references ...
1
vote
2answers
95 views

How to overwrite a file in the shell using < and > for the same file? [duplicate]

When I do my_script < filename.txt > filename.txt, the file is overwritten and truncated. Is there some way on the Unix command line to specify that redirection is not done concurrently, i.e. ...
1
vote
1answer
44 views

Occasional I/O Errors when attempting to transfer some files but others work fine

I'm running on Red Hat Enterprise Linux and when I attempt to transfer files through the file browser from one machine to a new directory on a server, I run into an issue where some files are able to ...
0
votes
1answer
95 views

update-initramfs: failed for /boot/initrd.img-4.15.0-112-generic with 1

descriptions: when i run dist-upgrade i ran into the problem with below detail 1 not fully installed or removed. After this operation, 0 B of additional disk space will be used. perl: warning: Setting ...
0
votes
0answers
30 views

How to show the output for the execution for any command and/or script in stdout and file but keeping the color in the stdout?

For some commands, in the terminal they print with some colors in the stdout, for example: git status mvn help:help -Ddetail=true gradle build Any Linux command (ls [-...], etc) Note: it applies for ...
0
votes
0answers
29 views

Server is slow, there's no workload, no apparent reason, only shutdown solves the problem

I manage a server that is now approximately 5 years old. It has Ubuntu Server installed. There had been a handful of times where suddenly it became super slow to the point it was basically unusable. ...
2
votes
0answers
17 views

Disabling user I/O during loop (temporarily)

I have a shell script that backs up my git repos that are all saved in one specific file. I wanted to add a cool "spinning load thingy" just for fun and that's pretty simple. The problem is ...
0
votes
0answers
28 views

How can I monitor whether disk activity is sychronous or asynchronous?

My Google-fu simply cannot find an answer to this. If I have a process with heavy I/O activity, how can I check whether it's using asynchronous or synchronous writes? (I want this information to ...
0
votes
0answers
25 views

NVME slower on host than in VM

I first noticed some strange performance hits when I switched from Vagrant to Docker for my local development setup. After investigating why my databases were slower in this new environment, I came to ...
0
votes
0answers
69 views

Why is SSHFS so much less stable than SSH?

If I'm on a very slow and/or slightly hiccupy internet connection, and I ssh foo@bar.baz, the connection will mostly stay up. If there's particularly bad lag, output (and echoed input) will take ...
0
votes
0answers
32 views

Could it be possible to throttle IOs using LD_PRELOAD?

I have a job on a managed Kubernetes cluster that is a bit IO intensive. Nothing crazy, it downloads about 1gb of data, writes it on /tmp folders, does some minor edits, compress and encrypts it, and ...
0
votes
2answers
143 views

Is It OK to use oflag=direct and conv=fsync together in dd?

My target device is a hard drive with 4096 bs. I want to bypass caches and write to storage directly or as soon as possible. Speed is not my primary concern: 30 hours is too much, but the difference ...
1
vote
1answer
39 views

bash I/O redirection - how to append to stderr

I have a script that loops over some big collection of data and performs some lenghty operations. Then i need to sort | uniq -c its output. So to let it know that its alive, I print a dot every N ...
1
vote
1answer
37 views

Unclear htop output when measuring IO usage of a dummy process

I write a simple test.sh script which only does this: while true; do cp ./target-file ./target-file.tmp; done wherein target-file is created using dd if=/dev/urandom of=target-file bs=1M count=100 and ...
1
vote
1answer
66 views

Linux reads ahead when concurrent reads done on same file

Linux performs readaheads (specified in /sys/block//queue/read_ahead_kb) when a file is read sequentially. Interested OS : Red Hat Linux Interested File System : xfs, ext4 What is the criteria for ...
0
votes
1answer
20 views

How does linux redirection manage fast input and slow output?

When I do this, $ cat ssd/file1 > hdd/file1 and read from ssd is faster than writing in hdd, what happens to the data that cat is reading? Is there any communication between input and output, which ...
0
votes
0answers
11 views

Measure I/O and network throughput performance (File Transfers)

I'm curious on what measures are being used to monitor the performance and stats of file transfers on Linux/Unix systems. The file transfer methods I was thinking of using was either rsync or sftp ...
0
votes
1answer
34 views

Sysdig does not show exit event for write syscall

I'm trying to capture write to stdout (or stderr), but apparently the actual data is related to the exit event. I wrote a simple C program that writes to stdout and stderr. #include <stdio.h> ...
1
vote
0answers
100 views

System keeps trying to access/read my laptop's empty CD drive

I'm using a 16-to-18-years-old laptop, a Clevo Zeus M3CW, on which I recently installed antiX Linux 19.3. The kernel version is 4.9.235 (compiled for 486). Now, whenever I boot, the kernel complains ...
1
vote
0answers
18 views

subshell and Input/Output Redirections questions

I have a couple of questions about the behavior of the shell i/o echo apple | echo banana The output here is only banana But in this command: echo apple | { echo banana; tail; } the output is ...
0
votes
0answers
52 views

fstrim - high usage of io on boot

I have one laptop Acer V3-771 with linux mint 20.1. (I format it like 2-3 months ago, but i was having the same problem before the format.) I am having issue with boot it get to slow and i think is ...
0
votes
0answers
13 views

MQ I/O schedulers, but I can't find multi-queue in the code

I have been looking at the mq-deadline source codes. Theoretically, mq-deadline utilizes a per-core sw queue. However, when I looked at the code, I can't find the multiple queues. They do have a per-...
0
votes
0answers
95 views

Why I don't see any increase in "Dirty" memory when writing to a file?

I read a bunch of stuff about "Write-back" mechanism today and wanted to try an experiment. EDIT: file downloaded from the Internet shows up in Dirty stats Based on the comments, I tried to ...
0
votes
0answers
24 views

Modify the generic block layer in a Linux system

For an university project, I need to modify a kernel layer in a Linux operating system. After modification I should prove performance improvements. Concrete I should modify the generic block layer and ...
0
votes
1answer
62 views

dd bs=X count=1 reads less than X bytes [duplicate]

I have dd from GNU coreutils 8.32. When I run { echo a; sleep 1; echo b; } | dd bs=4 count=1 then I get a 0+1 records in 0+1 records out 2 bytes copied, 2.0381e-05 s, 98.1 kB/s dd terminates during ...
2
votes
1answer
57 views

From the Linux kernel's point of view, how does a user program communicate with a CUDA GPU?

I am curious about how data is copied back and forth between CPU and GPU memories when I run a CUDA program. Specifically, I want to know how the Linux kernel is involved in this process. I had a few ...
1
vote
1answer
27 views

Does command line arguments and File I/O operation consume CPU Time potentially?

According to the definition of the CPU Time: CPU time (or CPU Execution time) is the time between the start and the end of execution of a given program. This time accounts for the time CPU is ...
0
votes
1answer
46 views

CPU bounded I/O performance issue with NVMe drive on Linux

I have an NVIDIA Jetson TX2 system with an NVMe drive with pcie gen2 x4 speed. The system is running a Linux 4.9 kernel, and I am experiencing issues with write performance. My aim is to write ...
1
vote
2answers
143 views

dmesg shows 0 512-byte logical blocks

Last week I unfortunatly dropped by external HDD drive and it wouldn't spin up anymore. I got other backups, so I thoughed I opened it up so see whats wrong. The read/write head was stuck and damaged, ...
0
votes
0answers
27 views

Is it more efficient to use file IO or swap space for large temporary storage?

(Cross-posted from SO.) I have a program that needs to produce a large quantity of temporary data (in the 10-100 gigabyte range) that I don't expect to fit in memory. The data is only used internally ...
0
votes
1answer
72 views

The difference between the udev rules and the proc sysfs for configuring the I/O scheduler? [closed]

Under which circumstances are the following config methods appropriate for use? "indirectly" use the udevadm via /etc/udev/rules.d/<iosched_file>: e.g. ACTION=="add|change", ...
2
votes
0answers
45 views

May a character device only handle blocking I/O?

Let's consider the following (imaginary) device: a clock which takes 1 second to query, then returns the current time. We want to write a character device driver for it, which supports read operations ...
0
votes
0answers
10 views

Some doubts of the I/O scheduler configuration with the partitioned sub-disks

Should I configure the same I/O scheduler with each partitioned sub-disks ? Could I configure the different I/O scheduler with each partitioned sub-disks ? # # # cat /sys/block/sdb/queue/...
1
vote
0answers
58 views

Writing to a device hangs, how can I troubleshoot this?

I have a teensy 3.6 as /dev/ttyACM0 I noticed that when I used serial.write() in python it would hang. I opened the Arduino IDE serial monitor and I can see the output. When I write in the serial ...
0
votes
0answers
14 views

Debugging slow disk speed: Why discrepancies between dd, raw disk, file systems, iozone?

I got a new 8TB Seagate disk which I connect via USB 3 and use with ZFS. When I am copying files via Samba (e.g., large 4GB file) I barely get morethan 4MB/s. When I am directly writing to the disk (e....
1
vote
1answer
32 views

Stream changes in available memory

I've come up with the following command to check changes in memory use: free -s 1 -h | awk 'NR%4==2' This shows output like this: Mem: 125G 32G 82G 404M 10G ...
-3
votes
1answer
146 views

Why performance degradation if read/write operations are not performed in multiples of the disk’s block size?

In Love's Linux System Programming, in CHAPTER 3 Buffered I/O Recall from Chapter 1 that the block is an abstraction representing the smallest unit of storage on a filesystem. Inside the kernel, all ...
0
votes
1answer
376 views

What value of "wa" from the top command is too high?

I have a server with 64 GB of ram, 16 CPU cores and an SSD drive. I am running a relational database on the machine. I would like to increase the writes to the database, and my guess would be that the ...
-1
votes
1answer
115 views

What is the path of data transfer when using memory mapped file?

Is there any difference between the data transfer paths using read()/write() and using mmap() on a file? What does "kernel" mean in https://stackoverflow.com/a/41419353? mmap doesn't ...
0
votes
1answer
45 views

How does preemption work during hardware I/O?

Say I have a device that uses SPI or another communication protocol to talk with my computer running Linux, Raspberry Pi for instance. For this communication to succeed, no interruptions are allowed, ...
0
votes
2answers
37 views

Get output and input into daemon processes

Background When a SIGHUP is sent to a running nginx master process to reconfigure, if there're password encrypted certificate key, nginx(specifically the openssl library) will ask for input, as in the ...
1
vote
1answer
414 views

Difference between BUFSIZ from stdio.h and block size from blockdev?

I'm reading The Unix Programming Environment (1984). In Chapter 7 (Unix System Calls) the authors note: While the number of bytes to be read or written is not restricted, the most cmmmon values are 1,...
2
votes
1answer
260 views

dm_crypt / kworker hogging IO and causing system freeze

I'm not sure how to debug this, but I've noticed if I am performing a task that requires a large amount of disk reads/write (such as updating a large postgres table) that periodically actual reads and ...
2
votes
1answer
111 views

Why does io_uring have a layer of indirection for the submission queue?

From the paper on io_uring, the submission ring contains indices into a submission array, where the submission event itself is stored. The documentation explains this layer of indirection as follows: ...
0
votes
1answer
31 views

Unclosed IO resources disrupt input at next shell prompt

Sometimes when I run a terminal program that spawns a child process, (e.g. IO.popen in a Ruby script), if the IO handle for the subprocess is not properly cleaned up (e.g. IO.close), then when the ...
1
vote
0answers
102 views

NGINX directio is not working (O_DIRECT flag in open(2) sys call )

Nginx configuration is location /conent/ { directio 200k; directio_alignment 4k; } Using SSD with ext4 file system with 4k block size, and journaling has been disabled, linux kernal version is 5.4., ...

1
2 3 4 5
12