I/O is an acronym for Input/Output, which refers to the mechanism or process for the transfer of information between one system or component and another.

learn more… | top users | synonyms

8
votes
2answers
970 views

What keeps one side of an rsync so busy?

I've got a Debian machine on my LAN serving as a backup server for the others. It has four HDDs combined into a software RAID 5 md device, on that an LVM, and on that btrfs. Backups are made using ...
3
votes
2answers
456 views

Running jobs in parallel on Ubuntu - I/O contention differences between Perl and Java

Apologies if this is off topic - it concerns the relative efficiencies of running I/O-heavy Perl/Java scripts in parallel on a Ubuntu system. I have written two simple versions of a file copy script ...
5
votes
3answers
461 views

File size of 60.0 PB is wrong. Can deleting it cause data loss?

While backing up some data (a 200 GB home directory) with rsync, I got an I/0 error for a particular file, after which rsync continued on "normally" with its backup. The problem source file showed as ...
2
votes
3answers
836 views

How can I read lines at a fixed speed?

I need to read a large log file and send it over a local network using (netbsd) netcat between two VMs on the same host workstation. I know that netcat has an interval, but as far as I can tell, the ...
17
votes
2answers
2k views

Portability of “> /dev/stdout”

Occasionally I need to specify a "path-equivalent" of one of the standard IO streams (stdin, stdout, stderr). Since 99% of the time I work with Linux, I just prepend /dev/ to get /dev/stdin, etc., ...
4
votes
4answers
962 views

script for logging all the stats for a particular process

I want to monitor CPU usage, disk read/write usage for a particular process, say ./myprocess. To monitor CPU top command seems to be a nice option and for read and write iotop seems to be a handy ...
3
votes
2answers
987 views

I/O statistics per user

I have access to an academic supercomputer where there is an NFS filesystem where our lab keeps our data. When I run iostats -nh I believe I am getting statistics for all users accessing all storage ...
2
votes
1answer
284 views

How do I record disk performance on slow hardware and then replay it to faster hardware and benchmark it?

I am currently performing various development tasks that are slow on my current system because of IO performance. Is there a way to record disk operations while performing IO consuming tasks, ...
21
votes
3answers
796 views

How can I limit the output speed of stdout?

I'm running CentOS 5.7 and I have a backup utility that has the option of dumping its backup file to stdout. The backup file is rather large (multiple gigabytes). The target is an SSHFS filesystem. To ...
5
votes
2answers
3k views

Write inside a socket open by another process in Linux

Is it possible on Linux for a process to write inside a socket open by another one? Let's say I open a connection to google.com using netcat: myuser@linux:~$ nc google.com 80 Now I can lookup for ...
2
votes
2answers
2k views

vmstat IO stats: why the high bo during a download?

On my RHEL 4 machine, I was downloading a huge file (2.5GB)..I was checking the output of vmstat and was especially curious about seeing the disk IO using bi and bo. I know iostat is a better tool ...
6
votes
2answers
404 views

Mysterious write IO

I am just reviewing the disk IO of a XEN VM server. I stumbled across a DomU that has constant write traffic. To narrow that traffic further down I logged in to the DomU (CentOS 5, 64 Bit) and ...
6
votes
1answer
867 views

SysBench highlighting abysmal disk write performance on white box vs Tier1 server

Have been testing white box system with SuperMicro X8DTL motherboard with various SATA hard disks such as 7,200RPM Seagate Constellation ES and 10,000 RPM Western Digital VelociRaptor. Have tested ...
4
votes
2answers
2k views

100.0%sy with poor disk performance

I have RHEL 5.4 kernel 2.6.18-164.el5 and I am experiencing very bad disk performance that happens randomly during working with Oracle. Nothing appears in the logs. When it happens, I see one CPU ...
3
votes
1answer
107 views

Copying a file that is written at the same time

This related question can't solve my exact problem, so I'm writing a similar question. My scenario is the following: a program (run by wine) writes to a file sized about 10-500 KB. About 1 KB ...
2
votes
1answer
125 views

In terminal, I can search to end of log faster than the data gets there

[The Question] Is there some way to have the pipe/tee/write combo write lines immediately? ... if it can be done, what is the trade-off? [The Backdrop] My script sends a keypress to the terminal, ...
3
votes
2answers
105 views

dealing with un-executable commands inside a shell-script

I am given an assignment like this: Write a Bash script that takes a *.tex file as command-line argument and checks if the file executes correctly without any errors. If it executes without any error ...
4
votes
1answer
1k views

Why do backgrounded processes sometimes stop spontaneously?

Sometimes, some time after I've backgrounded a process with bg in bash, when I press Enter in the same shell to redisplay the prompt (just to check that I'm still in bash when some output from the ...
3
votes
0answers
472 views

GNU/Linux, IO performance problems

I have some problems with my system performance. As I was adviced here, I used sar: $sar -u 5 and found out that the problems are caused by a low HDD performance, as I supposed (high %iowait). ...
2
votes
2answers
2k views

Make iotop show only the most disk-intensive item

My laptop (no VM, just plain Ubuntu with encrypted home) freezes for 3 minutes a few times per day. During these 3 minutes, the disk LED indicates intense disk activity, and I can't even move the ...
1
vote
2answers
113 views

Which root paths should reside on fast media?

Which paths under / should be mounted on fast media, and which can be mounted on slower media without significantly affecting system performance? My guess: Fast Partition: swap (2G) Partition: ...
9
votes
4answers
675 views

Utility to buffer an unbounded amount of data in a pipeline?

Is there a utility that I can stick in a pipeline to decouple read and write speeds? $ producer | buf | consumer Basically, I want a utility buf that reads its input as fast as possible, storing it ...
4
votes
0answers
159 views

Repeatably generate a read error for testing? [duplicate]

I'm trying to test the error handling in some software, in particular what happens when an error occurs reading from a file or pipe. Is there a simple way to send a certain amount of data to stdout ...
1
vote
1answer
343 views

BluRay Burner High I/O Wait State with UDF; Optimization Possible?

I've recently scaled back my home network, with one of the bigger changes being a switch-over to using a BD-RE drive for backup. With years of pictures and home video consuming about 25GB, and ...
3
votes
1answer
326 views

Removing redirection operator does not change output. Why?

I saw this usage of redirection somewhere, and thought it was a typo: grep root < /etc/passwd But after I run it, I saw that it gives the same output with grep root /etc/passwd: $ grep root ...
1
vote
1answer
651 views

iostat usage and output

When I run the iostat command like this: iostat -d I get a result something like this: Linux 2.6.18-238.el5 (mon01) 09/03/2011 Device: tps Blk_read/s Blk_wrtn/s Blk_read ...
1
vote
1answer
3k views

How can I specify that curl (via command line) overwrites a file if it already exists?

I'm using curl in this syntax: curl -o myfile.jpg http://mysite.com/myfile.jpg If I run this command twice, I get two files: myfile.jpg myfile-1.jpg How can I tell CURL that I want it to ...
6
votes
5answers
578 views

copy consumed output to stdout when stdout is redirected to a file

Given a command the output of which is being processed by, for example, grep I also want the actual output to be included in the log file for debugging purposes. For example if I have the following ...
3
votes
1answer
372 views

How race conditions affect reads and writes (that happen at the same time)

Let's say I open a file a for reading. What if an application, let's call it aWriter writes to this file in random intervals. Are there any possibilities of me receiving improper file contents if I ...
10
votes
1answer
1k views

Measuring disk I/O usage of a program

time is a brilliant command if you want to figure out how much CPU time a given command takes. I am looking for something similar that can measure the disk I/O of the program and any children. ...
7
votes
2answers
2k views

How to stop xargs from badly merging output from multiple processes?

I'm using xargs with the option --max-args=0 (alternatively -P 0). However, the output of the processes is merged into the stdout stream without regard for proper line separation. So I'll often end ...
7
votes
2answers
732 views

Piping output from a segfaulting program

I have a script that calls a program (specifically, ttf2afm, part of tetex 3.0) that sometimes segfaults and sometimes doesn't. The information I need is always printed out before it segfaults, but ...
20
votes
2answers
4k views

When is dd suitable for copying data? (or, when are read() and write() partial)

Short version: In what circumstances is dd safe to use for copying data, safe meaning that there is no risk of corruption due to a partial read or write? Long version — preamble: dd is often used to ...
12
votes
2answers
1k views

Weight-based block IO control for VMs

I use KVM to manage my virtual machines. I am trying to limit the resources provided for VMs. I am able to limit the CPU and memory allocated for VMs using libvirt/cgroups. Now I want to control the ...
11
votes
3answers
13k views

Find out what processes are writing to hard drive

On my Lenovo T400 and Ubuntu, the light for hard drive writing keeps flashing. I was wondering if in Linux it is possible to find out what processes are doing I/O to the hard drive? Just like by top, ...
1
vote
1answer
280 views

performance on Linux system when daisy-chaining multiple USB hard-drives on USB hubs?

What would the I/O access be like as I plug many USB drives into a Linux system with only 2 USB 2.0 ports and 1 USB 3.0 port? I expect I will have to have multiple levels of fan-out by plugging hubs ...
1
vote
1answer
145 views

What Linux tool will tell me if all the mounted disks share roughly the same bandwidth?

What Linux tool will tell me if all the mounted disks share roughly the same bandwidth? Can I do something like iostat for each of my /dev/sd* devices?
3
votes
1answer
323 views

Pluggin a bunch of USB hard drives into a Linux system?

I've got a Linux system with two USB 2.0 ports and one USB 3.0 port, and I would like to know how many USB hard drives I could plug into it. On the performance side, what would the I/O access be like ...
0
votes
1answer
278 views

How to test the difference between using disk encryption or not?

If I have a laptop, and I want to install an Ubuntu 11.04 on it, but disk performance would be an important thing, then how could I test the real differences between: - not using encryption - ...
4
votes
3answers
1k views

Intercept input and output from specific executable

I have a certain Linux executable that is called from a bigger application to perform some calculations. I want to modify the way the calculations are performed, for that I first need to understand ...
5
votes
2answers
2k views

Hight iowait while the wMB/s is low

I was indexing a MySQL table. It makes a high load on that computer. It seems due to a high iowait. But it also shows that the wMB/s is only 2.87. Isn't even a common SATA HDD able to handle more ...
3
votes
3answers
259 views

Changing the disk scheduler on the fly

Can the Linux disk scheduler be changed on the fly by writing to /sys/block/sda/queue/scheduler? Should applications (e.g. Mysql) be stopped / started when changing?
5
votes
5answers
433 views

performance monitoring

Is there some performace monitoring tool which would run in background gathering info about all system activity? Somethimes my system (Arch linux, 32 bit) slows down terribly and the top utility ...
4
votes
1answer
561 views

Does IO prioritise by the very nature of renicing a task?

The CFQ IO scheduler supports priorities though I am not sure that Deadline does (I believe not). The premise is that when I renice a task it gets a larger share of CPU under the Completely Fair ...
12
votes
5answers
45k views

How can I read line by line from a variable in bash?

I have a variable which contains multiline output of a command. What's the most effecient way to read the output line by line from the variable? For example: jobs="$(jobs)" if [ "$jobs" ]; then ...
7
votes
3answers
3k views

How can I monitor disk I/O in a particular directory?

I've got a few processes with a known name that all write to files in a single directory. I'd like to log the number of disk block reads and writes over a period (not just file access) to test whether ...
9
votes
2answers
8k views

Run process with higher priority

I want to run mplayer with higher priority than any other processes, including the IO-processes. How can I do that?
4
votes
1answer
553 views

“Split I/O”s on a Linux VM, disk alignment

I'm trying to figure out if there's some way I can tell if a system is being adversely affected by disk misalignment -- i.e. when I do a disk I/O operation, it actually needs to request extra blocks ...
15
votes
2answers
5k views

Why does high disk I/O reduce system responsiveness/performance?

I never quite understood why high disk I/O slowed the system so much. It's strange to me because I would expect the slow-down to affect only those processes dependent on the hard/optical drive data, ...
9
votes
1answer
1k views

What's the progress regarding improving system performance/responsiveness during high disk I/O?

Whenever there is high disk I/O, the system tends to be much slower and less responsive than usual. What's the progress on Linux kernel regarding this? Is this problem actively being worked on?