1

I noticed several times that pydf and df give a bit different disk usage values. For example:

$ df -h | grep sdb1
/dev/sdb1        11T  9.6T  705G  94% /mnt/disk
$ pydf | grep sdb1
/dev/sdb1   11T  10T  704G 88.6 [############.] /mnt/disk

pydf shows 10TB / 88.6% used and df -h shows 9.6TB / 94% used.

Even if I add inodes, which pydf -i shows:

$ pydf -i | grep sdb1
/dev/sdb1   349M 205k  349M  0.1 [.............] /mnt/disk

It's clearly too small amount of disk space to make up for 0.4TB of space.

Note - the difference is not in free disk space - it's a "live" system, so free disk space changed a little (704GB vs 705GB) between the commands - when I run it on idle system and use KB, it's the same number from pydf as from df)

Can anyone explain the difference?

Examples are from debian linux 10, but I've seen it on other systems too.

Additional information:

block size is 4096 bytes, and when forced to use block size, the comparison looks like this (edited to match "columns"):

$ (df --block-size=4096 && pydf --blocks) | grep sdb1
/dev/sdb1  2906334056 2568838216 191006112  94% /mnt/disk
/dev/sdb1  2906334056 2568838332 191005996  88.4 [#######.] /mnt/disk

again, since it's a "live system", the values are a bit different, but for this purpose I think it's ok to call those values (except percentage) the same.

As for what is on that filesystem - it's basicaly DVR (Digital Video Recorder) data disk - so there are large video clips, mostly 50-300MB, and almost no other (smaller) files.

7
  • out of interest : why do you compare pydf output to df -h one ? Shouldn't you compare df -h output to pydf -h ? Irrespective of this, you could probably post the output of cat /etc/pydfrc as well as cat ~/.pydfrc as part of your question
    – MC68020
    Commented Jul 28, 2022 at 13:02
  • @MC68020 technically, you're correct, but because pydf does "-h" somewhat implicitly, I think (you can clearly see the units are the same) it's ok for this case. As for configs, I didn't do anything to /etc/pydfrc, so it's "default values" (I can still post it if you confirm that it may be important), ~/.pydfrc does not exist.
    – Jan Spurny
    Commented Jul 28, 2022 at 13:20
  • What is the blocksize of the underlying filesystem ? What is the discrepancy when asking for number of blocks ? (df --block-size=the_actualsize vs pydf --blocks) ?
    – MC68020
    Commented Jul 28, 2022 at 13:33
  • @MC68020 - I added this to the question
    – Jan Spurny
    Commented Jul 28, 2022 at 14:48
  • 1
    One thing indeed curious is that df appears totally wrong in its used-blocks/total blocks computation. Possibly adding the reserved-blocks percentage (defaults on 5%) when pydf would not.
    – MC68020
    Commented Jul 28, 2022 at 14:59

1 Answer 1

1

From a df(1) man page one might read something like

Values are displayed in 512-byte per block counts

so one conclusion is that df measures by blocks and the other tool by something else. Both these tools are probably using some form of stat call (lstat, probably) to obtain information about the sizes of the files; over in a stat(2) man page one might notice

struct stat {
    ...
    off_t           st_size;    /* file size, in bytes */
    blkcnt_t        st_blocks;  /* blocks allocated for file */

that there is a file size in bytes and a number of blocks allocated for the file. This may suggest a test, what is the st_size and st_blocks for a file containing one byte?

#include <sys/stat.h>
#include <stdio.h>

int main(void) {
        struct stat buf;
        stat("file", &buf);
        printf("size\t%lld\nblocks\t%lld\n", buf.st_size, buf.st_blocks);
        return 0;
}

This script compiled to blocksize and with a suitable test file:

$ make blocksize
cc -O2 -pipe    -o blocksize blocksize.c
$ printf 1 > file
$ ./blocksize
size    1
blocks  4

shows that our one byte file has a size of 1, but on this system consumes 4 blocks of space. The block count and block size may vary, but this will account for the difference between a tool that counts by file size (st_size), and another that counts how many filesystem blocks (st_blocks) are used.

0

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .