2

I am trying to get the map of empty space on any partition in a filesystem-agnostic way. To do this I create a file that uses all of the empty space, then use the 'filefrag -e' command (e2fsprogs v1.42.9) to create a map of the space (on Ubuntu 14.04 Trusty, tested with kernels 3.16.0-67 and 4.1.20-040120, dosfstools v3.0.26-1).

This works for most filesystems, but for FAT filesystems specifically I am getting physical offsets beyond the size of the partition. Note the problem has now changed, please see the edit below.

$ dd if=/dev/zero of=temp.img bs=512 count=2048000
$ sudo losetup /dev/loop1 ./temp.img
$ sudo parted /dev/loop1 mklabel msdos
$ sudo parted /dev/loop1 mkpart primary fat32 2048s 1026047s
$ sudo blockdev --rereadpt /dev/loop1
$ sudo mkfs -t vfat /dev/loop1p1
$ sudo mount /dev/loop1p1 ./mnt
$ sudo cp somefile1 ./mnt
$ sudo cp somefile2 ./mnt
$ df -B 512 ./mnt
Filesystem     512B-blocks  Used Available Use% Mounted on
/dev/loop1p1       1023440 21232   1002208   3% ./mnt
$ sudo dd if=/dev/zero of=./mnt/emptyspace.zeros bs=512 count=1002208
$ df -B 512 ./mnt
Filesystem     512B-blocks    Used Available Use% Mounted on
/dev/loop1p1       1023440 1023440         0 100% ./mnt
$ sudo filefrag -b512 -e ./mnt/emptyspace.zeros 
Filesystem type is: 4d44
File size of ./mnt/emptyspace.zeros is 513130496 (1002208 blocks of 512 bytes)
 ext:     logical_offset:        physical_offset: length:   expected: flags:
   0:        0.. 1002207:     348688..   1350895: 1002208:    1350880: merged,eof
./mnt/emptyspace.zeros: 1 extent found
$ cat /proc/mounts
/dev/loop1p1 .../mnt vfat rw,relatime,fmask=0022,dmask=0022,codepage=437,
  iocharset=iso8859-1,shortname=mixed,errors=remount-ro 0 0
$ sudo umount /dev/loop1p1
$ sudo fsck /dev/loop1p1
fsck from util-linux 2.20.1
fsck.fat 3.0.26 (2014-03-07)
/dev/loop1p1: 4 files, 63965/63965 clusters
$ echo $?
0

(filefrag returns physical offsets relative to the start of the partition)

$ cat /sys/class/block/loop1p1/start 
2048
$ cat /sys/class/block/loop1p1/size
1024000

(sysfs start & size are in 512 byte sectors)

Clearly 1350895 is larger than 1024000. Is this a bug in the Linux vfat/fat implementation of the FIBMAP ioctl or is there another reason for this?

I note EmmaV posts a comment alluding to this problem in this question but there wasn't a definitive answer.

I have also been in touch with Theodore Ts'o (author of filefrag) and he has not indicated a known issue with filefrag.

EDIT: Further to this I have found the above problem is caused by a bug in e2fsprogs v1.42.9. A fix for this is available here which is first included in e2fsprogs v1.42.12. I have upgraded and tested and the output is very different.

However, I am still getting a problem with FAT filesystems. The offset is now inside the partition at least, but comparing the content of a file with the blocks returned by filefrag yields a difference. I have written a python script here for testing. I would be grateful for any feedback and suggestions on what the problem is.

Bonus points goes to the person that can tell me the problem with mkfs for btrfs! :)

  • I tried to reproduce it but stays within boundaries for me. What are the mount options according to /proc/mounts, and what is the filesystem size according to df ./emptyspace.zeros? It's possible for a filesystem to believe it's larger than the device... which version of filefrag? Does a Live CD with newer versions give the same results? – frostschutz Mar 27 '16 at 21:15
  • @frostschutz I've added answers to most of your questions in the edit. I'll try the latest kernel version too and report back.. – racitup Mar 27 '16 at 21:39
  • No idea, sorry; tried to reproduce with your exact device sizes, works for me. Does fsck complain in any way about this filesystem of yours? – frostschutz Mar 27 '16 at 22:05
  • @frostschutz Thanks for your help! Please could you confirm you have followed all steps above and get a different result? If so could you post what results you get and what kernel, filefrag and dosfstools version you are using? – racitup Mar 28 '16 at 14:07
1

I have been in touch with OGAWA Hirofumi and Theodore Ts'o and tested various kernels and e2fsprogs tags. The remaining problem is fixed in e2fsprogs v1.43-WIP from 2015 onwards. I believe this commit fixed the issue.

Full testing history and test script can be found here.

The moral of the story: don't bother using filefrag for FAT filesystems unless it says 1.43-WIP and 2015+ at the bottom of the man page.

I should also mention that hdparm --fibmap also has a buggy implementation in v9.43. You'll need at least v9.45 but I haven't thoroughly validated hdparm like filefrag.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.