0

I am looking for best way to determine how much each of my process has consumed in terms of disk space. As I see, there is only one option in front of me. That is to fetch the write_bytes from /proc/<pid>/io output from all the process and sum them up. But then I see a mis-match in terms of the total which I get by doing this to what df -h gives me.

  • 1
    None of those numbers in diskstat are disk usage. It contains information on I/O stats. – jordanm Dec 19 '17 at 16:27
  • The 8th column represents the total number of writes. But i do not know what each write size is. – Raks Dec 19 '17 at 16:30
  • I also looked at the /proc/<pid>/io stats and collected the write_bytes from all the PID's. That sum actually came upto 4GB which was not matching the 99GB disk space being used. Am i missing something here ? – Raks Dec 19 '17 at 16:32
  • @Jesse_b: My end goal is to figure out which process is leading to maximum disk utilization on a timely manner to define some data purge policies so that i do not run out of disk space at any given point in time. – Raks Dec 19 '17 at 16:51
  • Do you want to know the rate that each process is writing, or the cumulative amount that each process has written? Is a process writing 1MB/sec for a week better or worse than a process started a minute ago that is writing 100MB/sec? – Mark Plotnick Dec 19 '17 at 19:47
0

You could use iotop. This will show you disk usage in real time per process. Assuming you are using CentOS/Fedora, you can do.

$ sudo yum install iotop

Then just invoke with iotop.

  • iotop gives me the read/write throughput while i am interested in space consumption on disk. – Raks Dec 19 '17 at 17:27
  • Why can't you use iotop -a ? – Timothy Pulliam Dec 19 '17 at 17:34
  • How do i customize this to display just the top 20 consumers of disk space ? – Raks Dec 19 '17 at 17:47
0

I would use something like this pipe:

iotop -qqq -b -Paok -n 2 -d 30 | sort -n -k6 -r | head -n 20

Options:

-qqq : don't print headers and summary
-b : batch-mode
-P : show only processes, not all therads
-a : accumulated I/O
-k : print results in kilobytes
-n 2 : number of runs, from what I can understand, it must be >= 2 since the first one last less than 1 sec
-d 30 : delay between runs, aka total run in this case, approximately
| sort -n -k6 -r : numerically sort by 6th column (total writes) and reverse the order
| head -n 20 : show only the first 20 lines

Note : It shows only how much processes have written, not the total physical space used. If a process continuously write the same file of 20KB, once per sec, you end up having 20KB of space usage but in 30 sec a total write of 600KB. In this case I would monitor the directories my processes write into, with du

  • One thing which i wanted to know is if iotop actually starts incrementing counters from the time its run or will it also capture the already written data info on a per process basis no matter when i start iotop ? – Raks Dec 21 '17 at 5:30
  • @Raks, It starts collecting data from the time you run it but you can easily put a line in crontab (eg. @reboot /usr/sbin/iotop <arguments>) - or make a init service :) - to have a continuous log. – baselab Dec 21 '17 at 7:34
  • Is there a way to collect the number of bytes each process has deleted on disk over a period of time ? – Raks Jan 2 '18 at 5:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.