I/O is an acronym for Input/Output, which refers to the mechanism or process for the transfer of information between one system or component and another.

learn more… | top users | synonyms

69
votes
5answers
7k views

Redirecting stdout to a file you don't have write permission on

When you attempt to modify a file without having write permissions on it, you get an error: > touch /tmp/foo && sudo chown root /tmp/foo > echo test > /tmp/foo zsh: permission ...
43
votes
3answers
7k views

When is dd suitable for copying data? (or, when are read() and write() partial)

Short version: In what circumstances is dd safe to use for copying data, safe meaning that there is no risk of corruption due to a partial read or write? Long version — preamble: dd is often used to ...
7
votes
3answers
2k views

Construct a command by putting a string into a tty

I managed to do this echo -n "command" > /dev/tty1 The letters appear, and the cursor moves, but they are "ghosts" – if you hit Enter, nothing happens (they are not in stdin). Edit: In the ...
22
votes
3answers
14k views

Determining Specific File Responsible for High I/O

This is a simple problem but the first time I've ever had to actually fix it: finding which specific files/inodes are the targets of the most I/O. I'd like to be able to get a general system overview, ...
20
votes
6answers
85k views

How can I read line by line from a variable in bash?

I have a variable which contains multiline output of a command. What's the most effecient way to read the output line by line from the variable? For example: jobs="$(jobs)" if [ "$jobs" ]; then ...
23
votes
3answers
6k views

Measure pipe throughput in the shell

There is a shell command that allows to measure how fast the data goes through it, so you can measure the speed of output of commands in a pipe. So instead of: $ somecommand | anothercommand you ...
2
votes
1answer
1k views

-bash: /usr/sbin/useradd: Input/output error — What does this mean?

Today, on my Centos server I tried to add a user but got the following error: -bash: /usr/sbin/useradd: Input/output error How do I fix it?
24
votes
3answers
1k views

How can I limit the output speed of stdout?

I'm running CentOS 5.7 and I have a backup utility that has the option of dumping its backup file to stdout. The backup file is rather large (multiple gigabytes). The target is an SSHFS filesystem. To ...
23
votes
2answers
10k views

Can I watch the progress of a `sync` operation?

I've copied a large file to a USB disk mounted on a Linux system with async. This returns to a command prompt relatively quickly, but when I type sync, of course, it all has to go to disk, and that ...
26
votes
5answers
4k views

Why is iterating over a file twice faster than reading it into memory and computing twice?

I'm comparing the following tail -n 1000000 stdout.log | grep -c '"success": true' tail -n 1000000 stdout.log | grep -c '"success": false' with the following log=$(tail -n 1000000 stdout.log) echo ...
9
votes
1answer
1k views

What's the progress regarding improving system performance/responsiveness during high disk I/O?

Whenever there is high disk I/O, the system tends to be much slower and less responsive than usual. What's the progress on Linux kernel regarding this? Is this problem actively being worked on?
4
votes
2answers
254 views

Monitor a program's I/O

I have a program which spawns several child processes. I want to know which files this program and its children are creating, deleting, and modifying. inotifywait can tell me which files are being ...
27
votes
12answers
3k views

How can I tweak my Linux desktop to be more responsive?

I have the experience that Linux works fine until the physical memory is exhausted. As soon as swap space is used the performance is severely degraded and the GUI becomes unresponsive. This problem ...
8
votes
3answers
5k views

How can I monitor disk I/O in a particular directory?

I've got a few processes with a known name that all write to files in a single directory. I'd like to log the number of disk block reads and writes over a period (not just file access) to test whether ...
11
votes
5answers
1k views

Utility to buffer an unbounded amount of data in a pipeline?

Is there a utility that I can stick in a pipeline to decouple read and write speeds? $ producer | buf | consumer Basically, I want a utility buf that reads its input as fast as possible, storing it ...
6
votes
1answer
1k views

`unbuffer` or `stdbuf` for removing stdout buffering?

Is there a difference between unbuffer(1) and stdbuf(1)? From what I gather, unbuffer makes more than the "best effort" of calling the libc function set(X)buf at the beginning, and then letting things ...
2
votes
3answers
768 views

Amazon EC2 micro instance large number of IO requests

I'm trying to understand why I have a huge number of EBS IO request on my Amazon EC2 micro instance. I launched the instance roughly 6 days ago and has racked up over 4 million IO requests so far. ...
4
votes
2answers
729 views

What is the difference between > and >> (especially as it relates to use with the cat program)? [duplicate]

Suppose that I have a file called temp.txt. Using the cat program, I would like to add the contents of this file to the end of myfile.txt -- creating myfile.txt if it does not exist and appending to ...
12
votes
1answer
571 views

How do I go about tracking down IO usage in Linux and other *nix OS'?

Occasionally, something on my system starts hammering the disk. I would like to log data that would let me know about I/O usage per process, then I could go back to it later and see what keeps doing ...
2
votes
1answer
208 views

iotop showing 1.5 MB/s of disk write, but all programs have 0.00 B/s

I don't understand iotop output: it shows ~1.5 MB/s of disk write (top right), but all programs have 0.00 B/s. Why? The video was taken as I was deleting the content of a folder with a few millions ...
16
votes
3answers
14k views

How to Throttle per process I/O to a max limit?

I'm looking for a way to limit a processes disk io to a set speed limit. Ideally the program would work similar to this: $ limitio --pid 32423 --write-limit 1M Limiting process 32423 to 1 megabyte ...
10
votes
1answer
1k views

Know which process does I/O without iotop

On Linux, I want to know which process uses my disk I/O bandwith. I know I can use iotop but I have a machine where, for various reasons, iotop cannot be installed. How can I manually get that ...
6
votes
2answers
5k views

Write inside a socket open by another process in Linux

Is it possible on Linux for a process to write inside a socket open by another one? Let's say I open a connection to google.com using netcat: myuser@linux:~$ nc google.com 80 Now I can lookup for ...
26
votes
5answers
4k views

Portability of “> /dev/stdout”

Occasionally I need to specify a "path-equivalent" of one of the standard IO streams (stdin, stdout, stderr). Since 99% of the time I work with Linux, I just prepend /dev/ to get /dev/stdin, etc., ...
11
votes
2answers
2k views

Heavy write activity on SSD nukes system performance

I've noticed that when I do heavy write applications, the whole system slows down. To test this further I ran this to do a (relatively) low-CPU, high disk activity: john -incremental > file_on_SSD ...
8
votes
3answers
1k views

Why is I/O uninterruptible?

What is the rationale behind making I/O uninterruptible? What would be the negative consequences if a process handling I/O were allowed to handle a signal?
10
votes
1answer
3k views

How do I view the IO priority of a process?

How can I view the IO priority of a process? like to see for example if something has been ionice-ed.
5
votes
5answers
502 views

performance monitoring

Is there some performace monitoring tool which would run in background gathering info about all system activity? Somethimes my system (Arch linux, 32 bit) slows down terribly and the top utility ...
4
votes
2answers
582 views

why does blktrace only write blocks of 8?

I want to understand the I/O pattern a database is writing to disk to decide how many disks to use for best performance. To analyse the I/O pattern I want to use blktrace and I have to grok it first. ...
4
votes
0answers
161 views

Repeatably generate a read error for testing? [duplicate]

I'm trying to test the error handling in some software, in particular what happens when an error occurs reading from a file or pipe. Is there a simple way to send a certain amount of data to stdout ...
10
votes
4answers
4k views

Special File that causes I/O error

I want to automatically test if a piece of software reacts as expected if an essential SQLite DB file fails to be read (causing an I/O error). Exactly that happened some days ago at a client. We ...
8
votes
3answers
1k views

Harddrive I/O per Pid

is there any way to see the disk I/O per pid/process ? Monitorix shows me the following and correct me if I´m wrong, but it looks like that the harddrive is slowing down the whole system: UPDATE #1 ...
7
votes
2answers
2k views

Reduce bad block retry / wait times in Ubuntu

How can I reduce the IO wait time and retry times so that the OS doesn't continually try to write to a failing drive? I have an system that I use to make copies of demo content that gets loaned out ...
6
votes
2answers
2k views

iotop but for particular disk?

Is there a tool like iotop but for monitoring IO on specific volumes? (Or perhaps I missed a way to filter iotop?) If not is there any way to get at this information via (say)/proc? (dstat etc. ...
3
votes
2answers
80 views

How do I check which files are written by an application?

On my system, sar reports an average write to disk about 5MB/s. Using dstat I found that it's not so permanently, but there are peaks to 30-50MB/s, and then are sleeps. Using iotop i found that most ...
2
votes
1answer
380 views

Relationship between IO scheduler and cpu/process scheduler?

The concepts of IO scheduler and CPU scheduler confuse me. Below are my understanding: Linux uses CFS scheduler + nice values by default to schedule processes. Each process has an IO queue. There ...
6
votes
2answers
450 views

Mysterious write IO

I am just reviewing the disk IO of a XEN VM server. I stumbled across a DomU that has constant write traffic. To narrow that traffic further down I logged in to the DomU (CentOS 5, 64 Bit) and ...
5
votes
2answers
2k views

100.0%sy with poor disk performance

I have RHEL 5.4 kernel 2.6.18-164.el5 and I am experiencing very bad disk performance that happens randomly during working with Oracle. Nothing appears in the logs. When it happens, I see one CPU ...
3
votes
1answer
945 views

How race conditions affect reads and writes (that happen at the same time)

Let's say I open a file a for reading. What if an application, let's call it aWriter writes to this file in random intervals. Are there any possibilities of me receiving improper file contents if I ...
2
votes
1answer
221 views

How to track down the source of I/O spikes?

I am using Linux as a desktop, and keep getting seemingly random I/O Spikes. The machine becomes unusably slow. At first I thought it was just that I don't have enough memory. But looking at the ...
2
votes
3answers
322 views

Low level disk i/o on linux

Can a user level program do page level or block level I/O on SSDs? I've looked at disk devices, but am unsure if they provide this as they work only if the partition has a file system. UPDATE #1 I ...
1
vote
1answer
137 views

How to decrease the time the processes would wait (blocked) for disk reads (only to discover an error)?

Waiting too long when encountering an unreadable sector on a disk is really very annoying. (Especially if you are trying to read as much data as can be read from a disk with bad sectors, and do it ...
1
vote
2answers
3k views

How to know recently updated files

If I run iostat -x 1 I saw ocassionally large 5MB to 10MB writes. What files are being written? I want to check the recently created files with size over 5MB for example. How would I do so?
1
vote
1answer
453 views

Does symlinking from Drive A to Drive B affect I/O on Drive A?

I was wondering if IO through a symbolic link affects IO on the host drive even if it points to a seperate drive? My setup is, /dev/sdb2 1.8T 443G 1.3T 26% /home1 /dev/sda3 1.8T 1.7T ...
1
vote
1answer
316 views

performance on Linux system when daisy-chaining multiple USB hard-drives on USB hubs?

What would the I/O access be like as I plug many USB drives into a Linux system with only 2 USB 2.0 ports and 1 USB 3.0 port? I expect I will have to have multiple levels of fan-out by plugging hubs ...
1
vote
1answer
158 views

What Linux tool will tell me if all the mounted disks share roughly the same bandwidth?

What Linux tool will tell me if all the mounted disks share roughly the same bandwidth? Can I do something like iostat for each of my /dev/sd* devices?
0
votes
0answers
296 views

Is swapping the cause of high IO on my box?

I'm running a LAMP stack on an Amazon EC2 micro instance with 616 MB of memory. This seems sufficient to be able to run a few small websites with Apache, MySQL and PHP. I have one Wordpress site ...
0
votes
1answer
521 views

Does linux have something like supercache?

I installed http://www.superspeed.com/desktop/supercache.php for windows and the result is awesome. Every program I run is faster. Basically it uses the memory to cache data. Simple idea. I also use ...