18

I want to grok how fast a particular file is growing.

I could do

watch ls -l file

And deduce this information from the rate of change.

Is there something similar that would directly output the rate of growth of the file over time?

23

tail -f file | pv > /dev/null

But beware that it involves acually reading the file, so it might consume a bit more resources than something that watches just file size.

  • This does a nice job - just to save someone else a search or two - you'll need to yum install pv on a Centos/Redhat system to be able to do this ;-) – Ralph Bolton Jan 2 '18 at 16:19
9

progress (Coreutils progress viewer) or recent versions of pv can watch a file descriptor of a particular process. So you can do:

lsof your-file

to see what process ($pid) is writing to it and on which file descriptor ($fd), and do:

pv -d "$pid:$fd"

or:

progress -mp "$pid"
3

I have a little perl script that I put in my bash environment as a function:

fileSizeChange <file> [seconds]

Sleep seconds defaults to 1.

fileSizeChange() {
  perl -e '
  $file = shift; die "no file [$file]" unless -f $file; 
  $sleep = shift; $sleep = 1 unless $sleep =~ /^[0-9]+$/;
  $format = "%0.2f %0.2f\n";
  while(1){
    $size = ((stat($file))[7]);
    $change = $size - $lastsize;
    printf $format, $size/1024/1024, $change/1024/1024/$sleep;
    sleep $sleep;
    $lastsize = $size;
  }' "$1" "$2"
}
1

The following shell function monitors a file or directory and shows an estimate of throughput / write speed. Execute with monitorio <target_file_or_directory>. If your system doesn't have du, which could be the case if you are monitoring io throughput on an embedded system, then you can use ls instead (see comment in code)

monitorio () {
# show write speed for file or directory
    interval="10"
    target="$1"
    size=$(du -ks "$target" | awk '{print $1}')
    firstrun="1"
    echo ""
    while [ 1 ]; do
        prevsize=$size
        size=$(du -ks "$target" | awk '{print $1}')
        #size=$(ls -l "$1"  | awk '{print $5/1024}')
        kb=$((${size} - ${prevsize}))
        kbmin=$((${kb}* (60/${interval}) ))
        kbhour=$((${kbmin}*60))
        # exit if this is not first loop & file size has not changed
        if [ $firstrun -ne 1 ] && [ $kb -eq 0 ]; then break; fi
        echo -e "\e[1A $target changed ${kb}KB ${kbmin}KB/min ${kbhour}KB/hour size: ${size}KB"
        firstrun=0
        sleep $interval
    done
}

example use:

user@host:~$ dd if=/dev/zero of=/tmp/zero bs=1 count=50000000 &
user@host:~$ monitorio /tmp/zero
/tmp/zero changed 4KB 24KB/min 1440KB/hour size: 4164KB
/tmp/zero changed 9168KB 55008KB/min 3300480KB/hour size: 13332KB
/tmp/zero changed 9276KB 55656KB/min 3339360KB/hour size: 22608KB
/tmp/zero changed 8856KB 53136KB/min 3188160KB/hour size: 31464KB
^C
user@host:~$ killall dd; rm /tmp/zero
  • Thanks this worked great! I made a few small modifications if anyone is interested. My file transfer was spotty so I turned off stopping the script when the file size doesn't change, also added an optional second parameter to set the interval, and no longer printing the text on the first run since it's always 0: gist.github.com/einsteinx2/14a0e865295cf66aa9a9bf1a8e46ee49 – Ben Baron Aug 24 '18 at 16:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.