Tell me more ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.
up vote 7 down vote favorite
2

My document management software is doing a lot of IO and I would like to know which files it is accessing the most.

Is there a Linux tool that would give me the list of the top IO consuming files, like iotop but for files, every few seconds?

That could look like:

$ thetool
THRPUT   R/W/SWP FILE
40MB/s   write   /usr/alfresco/repo/1283421/1324928.doc
12MB/s   read    /usr/alfresco/cache/3928dh29f8if
11MB/s   read    /tmp/239398hf2f024f472.tmp

I looked in the man pages of iotop,lsof,strace and they do not seem to offer such a feature.

share|improve this question
You could write a post-processor to get this information from 'strace', but so far as I know, no such tool exists. (Such a tool would miss operations that took place through 'mmap'.) – David Schwartz Oct 27 '11 at 10:37

2 Answers

up vote 2 down vote accepted

I think your "number of bytes" metric is the wrong one. Consider two accesses. One reads 10MB from a file. The other reads every 512th byte of the file for the first 10MB. The "number of bytes" will be 512 times higher for the first access compared to the second. Yet they will both put precisely the same load on the I/O subsystem.

If you can accept "number of operations", which is just about as good or as bad as "number of bytes", then you have something you can actually measure. The inotifywatch program does this, and it's likely part of your distribution's inotify-tools package.

It will immediately tell you which files comprise the bulk of the accesses, and it will likely allow you to solve your actual problem.

share|improve this answer
iotop uses MB/s as a metric, is iotop wrong? +1 for your inotifywatch suggestion, the example in the man page is EXACTLY what I need, and what I described in my question. Thanks a lot! – Nicolas Raoul Oct 28 '11 at 1:51
Right, but 'iotop' can't identify files, only block devices. So it might help you identify the problem process and/or the problem filesystem/device, but not which files it's accessing. You may have to put pieces together from different tools. – David Schwartz Oct 28 '11 at 1:53
Hum, interesting, thanks! I am playing with inotifywatch now, and I will check what I can do. – Nicolas Raoul Oct 28 '11 at 1:57
up vote 0 down vote

First of all, to clarify: There is no such thing as "IO consuming files". Files are passive objects. It is programs that trigger IO (typically by reading/writing files) and they can access files more or less often. So the throughput you mention in regard to files would actually have not much sense. A useful information regarding files could be how frequently they are accessed and modified. This can be monitored using notifywait -m /some/file/or/directory (from inotify tools) or some more complex system like FAM or Gamin.

share|improve this answer
I would describe this "throughput" (other word might be more appropriate, sorry for my English) as the quantity of bytes that are read/written from/to a particular file during one second (sum for each process that accessed the file during that second). I hope it makes more sense. inotifywait,fam,gamin unfortunately only tell me "File X has been modified/read" which is interesting but not what I am looking for. – Nicolas Raoul Oct 27 '11 at 8:57

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.