I/O is an acronym for Input/Output, which refers to the mechanism or process for the transfer of information between one system or component and another.
1
vote
1answer
40 views
iotop but for particular disk?
Is there a tool like iotop but for monitoring IO on specific volumes? (Or perhaps I missed a way to filter iotop?) If not is there any way to get at this information via (say)/proc? (dstat etc. ...
8
votes
2answers
1k views
Heavy write activity on SSD nukes system performance
I've noticed that when I do heavy write applications, the whole system slows down. To test this further I ran this to do a (relatively) low-CPU, high disk activity:
john -incremental > file_on_SSD
...
4
votes
1answer
443 views
“Split I/O”s on a Linux VM, disk alignment
I'm trying to figure out if there's some way I can tell if a system is being adversely affected by disk misalignment -- i.e. when I do a disk I/O operation, it actually needs to request extra blocks ...
1
vote
0answers
37 views
Determine how much a process is using each disk?
How do I tell how much load a particular process is contributing to each of several available volumes/mounts? Seems that dstat only shows aggregate stats across all processes per disk, and iotop only ...
10
votes
1answer
477 views
How do I go about tracking down IO usage in Linux and other *nix OS'?
Occasionally, something on my system starts hammering the disk. I would like to log data that would let me know about I/O usage per process, then I could go back to it later and see what keeps doing ...
1
vote
1answer
354 views
I/O errors on hard disk on Linux boot
Here is a screenshot of booting Arch.
I guess the reason is that I force poweroff my Arch linux many times.
(I already force poweroff my Arch because my firefox flash plugin use too much memory to ...
2
votes
2answers
428 views
vmstat IO stats: why the high bo during a download?
On my RHEL 4 machine, I was downloading a huge file (2.5GB)..I was checking the output of vmstat and was especially curious about seeing the disk IO using bi and bo. I know iostat is a better tool ...
9
votes
6answers
15k views
How can I read line by line from a variable in bash?
I have a variable which contains multiline output of a command. What's the most effecient way to read the output line by line from the variable?
For example:
jobs="$(jobs)"
if [ "$jobs" ]; then
...
1
vote
0answers
687 views
How to set up ZFS with ZIL and L2ARC SSD?
I have been considering bcache and flashcache.
However my friend suggested me something called ZFS. In fact it says that formating disk with ZFS can actually do the exact same thing without kernel ...
3
votes
2answers
157 views
Storing thousands of files in one directory
I have a web-site that I'm checking for performance issues and bugs, and i came across a caching code that caches thousands of files into one single directory.
I understand that this is not good and ...
4
votes
2answers
116 views
Why is I/O uninterruptible?
What is the rationale behind making I/O uninterruptible? What would be the negative consequences if a process handling I/O were allowed to handle a signal?
5
votes
2answers
129 views
How to improve IO when there is a lot of random read and write?
My scripts uses tons of read and writes. Most of them are small and random. I want Linux to wait 10 seconds before writing any data to the disk.
How would this be achieved?
I think Linux can already ...
0
votes
1answer
144 views
Does linux have something like supercache?
I installed http://www.superspeed.com/desktop/supercache.php for windows and the result is awesome.
Every program I run is faster.
Basically it uses the memory to cache data. Simple idea. I also use ...
6
votes
3answers
1k views
How can I monitor disk I/O in a particular directory?
I've got a few processes with a known name that all write to files in a single directory. I'd like to log the number of disk block reads and writes over a period (not just file access) to test whether ...
13
votes
2answers
3k views
Why does high disk I/O reduce system responsiveness/performance?
I never quite understood why high disk I/O slowed the system so much. It's strange to me because I would expect the slow-down to affect only those processes dependent on the hard/optical drive data, ...
1
vote
2answers
550 views
Make iotop show only the most disk-intensive item
My laptop (no VM, just plain Ubuntu with encrypted home) freezes for 3 minutes a few times per day. During these 3 minutes, the disk LED indicates intense disk activity, and I can't even move the ...
0
votes
1answer
56 views
Strange IOStat xk. Moving a mere 40GB data is too slow. Can anyone explain?
iostat -xk 5
avg-cpu: %user %nice %system %iowait %steal %idle
0.99 0.00 1.10 11.01 0.00 86.89
Device: rrqm/s wrqm/s r/s w/s rkB/s wkB/s ...
0
votes
2answers
79 views
is writing to slow storage in nohup mode blocking the process?
If I have a process writing to nohup and the storage where it writes to is slow, does it block the process which is running in nohup? For example:
process A is running in nohup, it needs to write 10 ...
1
vote
2answers
133 views
How to know recently updated files
If I run iostat -x 1 I saw ocassionally large 5MB to 10MB writes.
What files are being written?
I want to check the recently created files with size over 5MB for example.
How would I do so?
2
votes
1answer
89 views
IOStat report large ocassional writes but find reports no big file
root@host [/home4]# find . -type f -size +2000k -exec ls -lh {} \; | awk '{ print $9 ": " $5 }'
...
1
vote
1answer
101 views
iostat report huge writes to drives that's not even mounted
root@host [~]# fsck /home2
fsck from util-linux-ng 2.17.2
e2fsck 1.41.12 (17-May-2010)
/dev/sdb1: clean, 6018617/91578368 files, 54524459/366284000 blocks
root@host [~]# fsck /home4
fsck from ...
5
votes
2answers
3k views
root user denied access to .gvfs in rsnapshot?
I was running rsnapshot as root and I got the following error. Why would this happen? what is .gvfs?
rsnapshot weekly slave-iv
...
1
vote
1answer
616 views
Which drive had a “journal commit I/O error”?
I received a message:
kernel:[123456.789012] journal commit I/O error
Which disk drive had the journal error?
4
votes
2answers
95 views
Determine which process is taking most of disk bandwidth?
When the HDD indicator is blinking (for a long period), how could I know which process is taking most disk bandwidth?
1
vote
1answer
83 views
Does symlinking from Drive A to Drive B affect I/O on Drive A?
I was wondering if IO through a symbolic link affects IO on the host drive even if it points to a seperate drive?
My setup is,
/dev/sdb2 1.8T 443G 1.3T 26% /home1
/dev/sda3 1.8T 1.7T ...
4
votes
4answers
424 views
script for logging all the stats for a particular process
I want to monitor CPU usage, disk read/write usage for a particular process, say ./myprocess.
To monitor CPU top command seems to be a nice option and for read and write iotop seems to be a handy ...
10
votes
1answer
696 views
Meaning of colours in htop
Given the option:
[ ] Detailed CPU time (System/IO-Wait/Hard-IRQ/Soft-IRQ/Steal/Guest)
In htop (Version 1.0.1) what are the meanings of the colours?
In the following image, I'm seeing rather a lot ...
6
votes
2answers
668 views
Can I watch the progress of a `sync` operation?
I've copied a large file to a USB disk mounted on a Linux system with async. This returns to a command prompt relatively quickly, but when I type sync, of course, it all has to go to disk, and that ...
2
votes
1answer
746 views
I/O error but no bad block is reported in syslog
I got errors like this in my syslog, but there's no bad block or sector defined in kernel report, I can't just replace the HDD right now and I prefer to reallocate the bad blocks for now, so I need ...
4
votes
2answers
1k views
How to Throttle per process I/O to a max limit?
I'm looking for a way to limit a processes disk io to a set speed limit. Ideally the program would work similar to this:
$ limitio --pid 32423 --write-limit 1M
Limiting process 32423 to 1 megabyte ...
24
votes
12answers
2k views
How can I tweak my Linux desktop to be more responsive?
I have the experience that Linux works fine until the physical memory is exhausted. As soon as swap space is used the performance is severely degraded and the GUI becomes unresponsive.
This problem ...
0
votes
1answer
96 views
What is the difference between > and >> (especially as it relates to use with the cat program)?
Suppose that I have a file called temp.txt. Using the cat program, I would like to add the contents of this file to the end of myfile.txt -- creating myfile.txt if it does not exist and appending to ...
4
votes
2answers
2k views
Write inside a socket open by another process in Linux
Is it possible on Linux for a process to write inside a socket open by another one?
Let's say I open a connection to google.com using netcat:
myuser@linux:~$ nc google.com 80
Now I can lookup for ...
5
votes
1answer
421 views
What keeps one side of an rsync so busy?
I've got a debian machine in my lan serving as a backup server for the others. It has four HDDs combined into a software RAID 5 md device, on that an lvm, and on that a btrfs. Backups are made using ...
5
votes
3answers
258 views
File size of 60.0 PB is wrong. Can deleting it cause data loss?
While backing up some data (a 200 GB home directory) with rsync, I got an I/0 error for a particular file, after which rsync continued on "normally" with its backup. The problem source file showed as ...
3
votes
2answers
200 views
Running jobs in parallel on Ubuntu - I/O contention differences between Perl and Java
Apologies if this is off topic - it concerns the relative efficiencies of running I/O-heavy Perl/Java scripts in parallel on a Ubuntu system.
I have written two simple versions of a file copy script ...
2
votes
3answers
221 views
How can I read lines at a fixed speed?
I need to read a large log file and send it over a local network using (netbsd) netcat between two VMs on the same host workstation.
I know that netcat has an interval, but as far as I can tell, the ...
1
vote
0answers
73 views
Changing I/O settings from serial port to LCD&keyboard
I have a prebuilt image of Linux (uImage-da850-omapl138-evm.bin) from Arago Project.
So uImage and root filesystem are situated on SD card of my device (OMAP-L138 Experimenter Kit). After Linux is ...
3
votes
2answers
455 views
I/O statistics per user
I have access to an academic supercomputer where there is an NFS filesystem where our lab keeps our data.
When I run iostats -nh I believe I am getting statistics for all users accessing all storage ...
11
votes
2answers
513 views
Portability of “> /dev/stdout”
Occasionally I need to specify a "path-equivalent" of one of the standard IO streams (stdin, stdout, stderr). Since 99% of the time I work with Linux, I just prepend /dev/ to get /dev/stdin, etc., ...
2
votes
1answer
160 views
How do I record disk performance on slow hardware and then replay it to faster hardware and benchmark it?
I am currently performing various development tasks that are slow on my current system because of IO performance.
Is there a way to record disk operations while performing IO consuming tasks, ...
18
votes
3answers
525 views
How can I limit the output speed of stdout?
I'm running CentOS 5.7 and I have a backup utility that has the option of dumping its backup file to stdout. The backup file is rather large (multiple gigabytes). The target is an SSHFS filesystem. To ...
4
votes
3answers
760 views
Intercept input and output from specific executable
I have a certain Linux executable that is called from a bigger application to perform some calculations. I want to modify the way the calculations are performed, for that I first need to understand ...
6
votes
2answers
264 views
Mysterious write IO
I am just reviewing the disk IO of a XEN VM server. I stumbled across a DomU that has constant write traffic.
To narrow that traffic further down I logged in to the DomU (CentOS 5, 64 Bit) and ...
3
votes
3answers
154 views
Changing the disk scheduler on the fly
Can the Linux disk scheduler be changed on the fly by writing to /sys/block/sda/queue/scheduler? Should applications (e.g. Mysql) be stopped / started when changing?
5
votes
1answer
577 views
SysBench highlighting abysmal disk write performance on white box vs Tier1 server
Have been testing white box system with SuperMicro X8DTL motherboard with various SATA hard disks such as 7,200RPM Seagate Constellation ES and 10,000 RPM Western Digital VelociRaptor.
Have tested ...
11
votes
2answers
722 views
Weight-based block IO control for VMs
I use KVM to manage my virtual machines. I am trying to limit the resources provided for VMs. I am able to limit the CPU and memory allocated for VMs using libvirt/cgroups. Now I want to control the ...
3
votes
2answers
995 views
100.0%sy with poor disk performance
I have RHEL 5.4 kernel 2.6.18-164.el5 and I am experiencing very bad disk performance that happens randomly during working with Oracle.
Nothing appears in the logs.
When it happens, I see one CPU ...
3
votes
1answer
98 views
Copying a file that is written at the same time
This related question can't solve my exact problem, so I'm writing a similar question.
My scenario is the following: a program (run by wine) writes to a file sized about 10-500 KB. About 1 KB ...
6
votes
2answers
406 views
Piping output from a segfaulting program
I have a script that calls a program (specifically, ttf2afm, part of tetex 3.0) that sometimes segfaults and sometimes doesn't. The information I need is always printed out before it segfaults, but ...
