How can I identify the total files open by userid

Question

Yesterday one of the users on our system had a poorly formed script that consumed all the space in one of our file systems. Through a lot of trial and error, I was able to identify the user and kill the processes causing the problem.

Is there a simple way to identify such processes in the future?

I think the output from the lsof /file-system command will display each open file with USER, PID, and file size and I suppose I could write a program to parse the output, but I'm curious if there is another command to do this. I really want a report of total file size by user, just to help focus if this problem happens again.

I'm on Solaris 10, if that's relevant. And I'm not the System Admin; I'm probably a "power user" and coordinate things for our department end-users.

Answer 1

lsof is only able to report files currently open by processes. That won't help you if the data has already been written out to disk or for tracking IO.

If there's a lot of IO happening at the time you are looking, you might be able to catch the offending process with iotop from the DTraceToolkit

Size on disk is a bit harder to do adhoc reporting on due to the time a scan across a file system takes. Anything outside what the underlying filesystem tracks is a bit time consuming, especially if you need to report it regularly. Reports won't do anything to stop the problem either so it's probably best to use file system and user quotas to restrict how much data can be written where. Once you have user quota's setup you will be able to report usage for a particular user more easily with the in built reporting.

As a one off report (without quotas) you could run something like this to get an estimate of space used in MB per user.

sudo find /path -type f -ls | perl -lane ' $total{ $F[4] } += $F[6]; 
END { map { printf "%-8s %d\n", $_, $total{$_}/2**20; } keys %total; }'

This will be slightly off due to space really being allocated in blocks and hard links will falsely inflate values, but good enough to catch outliers if you can wait for the find to complete :). Sorry it's a bit perlish.