3

XFS supports copy on write (CoW), so it is not entirely clear what du will say if some of the bytes are shared across files. I'd like to find a way to check how much disk space a folder uses, not counting shared bytes multiple times, i.e. the real usage on disk.

Neither xfs_estimate nor du seem to do what I need:

$ mkdir testfolder
$ cd testfolder 
$ dd if=/dev/zero of=testfile  bs=1M count=500 status=progress
500+0 records in
500+0 records out
524288000 bytes (524 MB, 500 MiB) copied, 0,158889 s, 3,3 GB/s
$ cp --reflink=always testfile testfile2                      
$ xfs_estimate .                         
. will take about 1004,4 megabytes
$ du -hs .      
1000M   .

What I expect is that some tool says that this folder uses only 500MB.

df shows that free disk spaces is reduced by 500MB when using a plain cp, but not at all when doing a cp --reflink=always. So reflinking seems to work, but df is not helpful in practice, because the disk is huge and I want to check the real size of a quite small folder.

I assume this might be a valid question for BTRFS too. But in my case, I need a solution which works for XFS.

2
  • What I expect is that some tool says that this folder uses only 500MB. And what should it report if the data blocks for the files in that directory are all shared with files in other directories? Jan 7 at 11:29
  • @AndrewHenle well, I don't know, still 500MB, since I am interested in the size of this specific folder on the disk? Would be nice to have a second way to find these files in other directories.
    – lumbric
    Jan 8 at 10:04
3

It feels like there should be a tool that does this by default but I don't remember if there is one.

You can query file extents using filefrag (generic, FIEMAP ioctl), or using xfs_bmap (XFS specific). That way you could choose to count shared extents (duplicates) only once (or not at all).

# filefrag -e -k testfile
Filesystem type is: 58465342
File size of testfile is 5242880 (5120 blocks of 1024 bytes)
 ext:     logical_offset:        physical_offset: length:   expected: flags:
   0:        0..    5119:         96..      5215:   5120:             last,shared,eof
testfile: 1 extent found

In this example, filefrag knows and shows that the extent is shared (anywhere in the filesystem, not necessarily within that directory), xfs_bmap does not:

# xfs_bmap -l testfile
testfile:
    0: [0..10239]: 192..10431 10240 blocks

But basically that's the key ingredient you can use to script it on your own.

Show all possible shared extents:

# find . -xdev -type f -exec filefrag -e -k {} + | grep shared
   0:        0..    5119:         96..      5215:   5120:             last,shared,eof
   0:        0..    5119:       5216..     10335:   5120:             last,shared,eof
   0:        0..    5119:         96..      5215:   5120:             last,shared,eof

Shared (duplicate within directory) extents using xfs_bmap:

# find . -xdev -type f -exec xfs_bmap -l {} + | grep 'blocks$' | grep -v ': hole' | sort | uniq -d
0: [0..10239]: 192..10431 10240 blocks

Note that xfs_bmap uses 512 byte per block whereas filefrag uses 1024 bytes (with -k option) or whatever the filesystem blocksize is (like 4096 bytes).

Shared duplicate extents using filefrag:

# find . -xdev -type f -exec filefrag -ek {} + | grep shared | sort | uniq -d
   0:        0..    5119:         96..      5215:   5120:             last,shared,eof

So in this case you'd have to substract 5120 from the du -cks . result.

3
  • This is just an example that covers the simplest cases. With filefrag in particular there can be duplicate extents with different flags (if a shared file was truncated or appended to, it might keep the shared but lose the last,eof flags). For a proper solution, you'll have to script it on your own according to your intentions. With shared extents, it's often not clear when to count and when not to count them. Jan 7 at 22:43
  • Oh great! That sounds like you need to know quite some low-level stuff about filesystems, to get such a script actually working correctly. Can you give me an example when the result could be ambiguous? I guess in simplest case it should simply return the number of bytes required by this folder (imagine all other data on the disk was deleted, how many bytes are occupied?).
    – lumbric
    Jan 8 at 10:09
  • 1
    @lumbric How many bytes if everything else was deleted is a different answer from how many bytes would be freed if you deleted just this directory. for incremental backups most shared extents would not be within the same directory but outside of it, etc. If you use copy-on-write a lot and files get modified in place, shared extents can get quite fragmented. For figuring out shared extent relationships completely you'd probably have to map out the entire filesystem, not just some part of it. So it depends on what you really want to know, there's not just the one answer... Jan 8 at 12:02
0

I made a solution for this over on the Veeam community pages:

Inherency:

Since we bill by actual disk usage for our client utilizing (Veeam) CloudConnect, the existing reports are non-functional for reporting and billing (They report pre-deduplication/reflink data). It took a while to figure out, but there is a way to calculate actual disk space used on a per directory on an immutable repository.

Solution:

Using: https://community.veeam.com/blogs-and-podcasts-57/check-reflink-and-spared-space-on-xfs-repositories-244

I have extrapolated a script that will give the disk usage of each folder (IE: client) on an immutable repo. This isn't "Data used"; that is what Veeam reports. This is "Disk Used". The actual size on disk after reflinks (duplicated data is only counted once).

A note about this script; it appears that the original blog entry is wrong on the size of a block. They attribute it to 4096... which is true... on disk... but the utility used explicitly gives the information in block sizes of 512: https://linux.die.net/man/8/xfs_bmap "units of 512-byte blocks"

To use this script, we use a cron task and pipe the output to a mail client on the repo itself (IE: script.bash 2>&1 | mail -s “Immutable storage report for $HOSTNAME” someemail@email.place)

#!/bin/bash

for clientDir in `find /backups/disk-01/backups/ -mindepth 1 -maxdepth 1 -type d`
do
    echo $clientDir
    clientSpaceUsed=$(find $clientDir/*/* -xdev -type f -exec xfs_bmap -l {} + | awk '{ print $3 " " $4 }' | sort -k 1 | uniq | awk '{ print $2 }' | grep -Eo '[0-9]{1,7}' | paste -sd+ | bc | awk '{print $1*512/1024/1024/1024}')
    #block sizes of 512 bytes.  Divided by 1024 for KB.  Divided by 1024 for MB. Divided by 1024 for GB.
    echo "$clientSpaceUsed GB"
done

To break down how this works:

For each client directory in “/backups/disk-01/backups/”

output the directory being reported on

run xfs_bmap -l (this tells us all about the blocks in question) Take columns 3 and 4 (now becomes column 1 and 2, the rest are discarded) sort by column 1 remove duplicate rows of data (reflinks for fast cloning; keeps a single copy of the data for counting purposes) Select only column 2 (now becomes column 1)

remove anything other than numbers

Add those numbers together

multiple by block size (512)

divide by 1024 (now KB)

divide by 1024 (now MB)

divide by 1024 (now GB)

output text

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.