1

I am looking for best way to determine how much each of my process has consumed in terms of disk space. As I see, there is only one option in front of me. That is to fetch the write_bytes from /proc/<pid>/io output from all the process and sum them up. But then I see a mis-match in terms of the total which I get by doing this to what df -h gives me.

9
  • 1
    None of those numbers in diskstat are disk usage. It contains information on I/O stats.
    – jordanm
    Dec 19, 2017 at 16:27
  • The 8th column represents the total number of writes. But i do not know what each write size is. Dec 19, 2017 at 16:30
  • I also looked at the /proc/<pid>/io stats and collected the write_bytes from all the PID's. That sum actually came upto 4GB which was not matching the 99GB disk space being used. Am i missing something here ? Dec 19, 2017 at 16:32
  • @Jesse_b: My end goal is to figure out which process is leading to maximum disk utilization on a timely manner to define some data purge policies so that i do not run out of disk space at any given point in time. Dec 19, 2017 at 16:51
  • Do you want to know the rate that each process is writing, or the cumulative amount that each process has written? Is a process writing 1MB/sec for a week better or worse than a process started a minute ago that is writing 100MB/sec? Dec 19, 2017 at 19:47

2 Answers 2

1

I would use something like this pipe:

iotop -qqq -b -Paok -n 2 -d 30 | sort -n -k6 -r | head -n 20

Options:

-qqq : don't print headers and summary
-b : batch-mode
-P : show only processes, not all therads
-a : accumulated I/O
-k : print results in kilobytes
-n 2 : number of runs, from what I can understand, it must be >= 2 since the first one last less than 1 sec
-d 30 : delay between runs, aka total run in this case, approximately
| sort -n -k6 -r : numerically sort by 6th column (total writes) and reverse the order
| head -n 20 : show only the first 20 lines

Note : It shows only how much processes have written, not the total physical space used. If a process continuously write the same file of 20KB, once per sec, you end up having 20KB of space usage but in 30 sec a total write of 600KB. In this case I would monitor the directories my processes write into, with du

3
  • One thing which i wanted to know is if iotop actually starts incrementing counters from the time its run or will it also capture the already written data info on a per process basis no matter when i start iotop ? Dec 21, 2017 at 5:30
  • @Raks, It starts collecting data from the time you run it but you can easily put a line in crontab (eg. @reboot /usr/sbin/iotop <arguments>) - or make a init service :) - to have a continuous log.
    – baselab
    Dec 21, 2017 at 7:34
  • Is there a way to collect the number of bytes each process has deleted on disk over a period of time ? Jan 2, 2018 at 5:22
-1

You could use iotop. This will show you disk usage in real time per process. Assuming you are using CentOS/Fedora, you can do.

$ sudo yum install iotop

Then just invoke with iotop.

3
  • iotop gives me the read/write throughput while i am interested in space consumption on disk. Dec 19, 2017 at 17:27
  • Why can't you use iotop -a ? Dec 19, 2017 at 17:34
  • How do i customize this to display just the top 20 consumers of disk space ? Dec 19, 2017 at 17:47

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .