10

Is there any command that will output the beginning and ending blocks of any file?

  • 1
    what file system type: ext2,3,4; btrfs; xfs; zfs, etc...? – BeowulfNode42 Dec 27 '13 at 18:46
  • @BeowulfNode42: ext4, ntfs, fat32 are the ones I often deal with... so preferably for these three... – precise Dec 27 '13 at 18:53
  • The question should be improved (be more precise): My first answer would have been some program that opens the file, reads the first block, then seeks to the last block and read that, too. So what is "output" of a block? The block's content, the block's logical address (inside the file, inside the file system, inside the partition, or inside the block device), or the block's physical address (gets interesting if the disk is part of some RAID or LVM). The answers seem much better than the question. – U. Windl Dec 28 '19 at 17:38
16

hdparm

I'm not 100% sure this is what you're looking for but I believe you can do this using the command hdparm, specifically with its --fibmap switch.

excerpt

   --fibmap
          When  used,  this  must  be the only option given.  It requires a 
          file path as a parameter, and will print out a list of the block 
          extents (sector ranges) occupied by that file on disk.  Sector 
          numbers are  given as absolute LBA numbers, referenced from sector 
          0 of the physical device rather than from the partition or 
          filesystem.  This information can then be used for a variety of 
          purposes,  such  as examining the degree of fragmenation of larger 
          files, or determining appropriate sectors to deliberately corrupt 
          during fault-injection testing procedures.

          This option uses the new FIEMAP (file extent map) ioctl() when 
          available,  and  falls  back  to  the older  FIBMAP (file block 
          map) ioctl() otherwise.  Note that FIBMAP suffers from a 32-bit 
          block-number interface, and thus not work beyond 8TB or 16TB.  
          FIBMAP is also very slow, and  does  not  deal well  with  
          preallocated uncommitted extents in ext4/xfs filesystems, unless a 
          sync() is done before using this option.

Example

Say we have a sample file.

$ echo "this is a test file" > afile

Now when we run hdparm.

$ sudo hdparm --fibmap afile 

afile:
 filesystem blocksize 4096, begins at LBA 0; assuming 512 byte sectors.
 byte_offset  begin_LBA    end_LBA    sectors
           0  282439184  282439191          8

filefrag

Another nice method for finding out a file's beginning & ending blocks is filefrag. You'll need to use appropriate switches though, to get the desired output. One upside of this tool over hdparm is that any user can run it, so no sudo is required. You'll need to use the -b512 switch so that outputs are displayed in 512 byte blocks. Also we need to tell filefrag to be verbose.

Example

$ filefrag -b512 -v afile
Filesystem type is: ef53
File size of afile is 20 (8 block of 512 bytes)
 ext:     logical_offset:        physical_offset: length:   expected: flags:
   0:        0..       7:  282439184.. 282439191:      8:             eof
afile: 1 extent found

debugfs

A third method for getting a file's LBAs is to make use of debugfs. This method will require a little math, but I thought it important to show how one can convert from the extents value reported by debugfs to LBAs, for those that might be curious.

So let's start with the file's inode.

$ ls -i afile
6560281 afile

NOTE: We could also use the file's name within debugfs but for this demonstration I'm going to use the inode instead.

Now let's get the stat information via debugfs about our inode.

$ sudo debugfs -R "stat <6560281>" /dev/mapper/fedora_greeneggs-home
debugfs 1.42.7 (21-Jan-2013)
Inode: 6560281   Type: regular    Mode:  0664   Flags: 0x80000
Generation: 1999478298    Version: 0x00000000:00000001
User:  1000   Group:  1000   Size: 20
File ACL: 0    Directory ACL: 0
Links: 1   Blockcount: 8
Fragment:  Address: 0    Number: 0    Size: 0
 ctime: 0x52be10c3:a640e994 -- Fri Dec 27 18:44:03 2013
 atime: 0x52bff8a1:a9f08020 -- Sun Dec 29 05:25:37 2013
 mtime: 0x52be0fe7:18a2f344 -- Fri Dec 27 18:40:23 2013
crtime: 0x52be0dd8:64394b00 -- Fri Dec 27 18:31:36 2013
Size of extra inode fields: 28
Extended attributes stored in inode body: 
  selinux = "unconfined_u:object_r:user_home_t:s0\000" (37)
EXTENTS:
(0):35304898

The important information is in the extents section. These are actually filesystem blocks that are being used by this inode. We just need to convert them to LBA. We can do this through the following equation.

NOTE: Assuming that our filesystem uses 4k block sizes and that underlying hardware uses 512 byte units, we need to multiply the exents by 8.

beginning LBA = (BEGIN EXTENT) * 8
ending LBA    = (((ENDING EXTENT) + 1) * 8) - 1

Example

So in our example our beginning and ending extent is the same, since our file fits within a single extent.

beginning LBA = 35304898 * 8             = 282439184
ending LBA    = ((35304898 + 1) * 8) - 1 = 282439191

So our LBAs are 282439184..282439191.

References

  • these are some links... thanks for the answer and links... – precise Dec 28 '13 at 5:02
  • 2
    @hash - they're the remnants of me trying to find 2 other methods for determining the LBAs. 8-). I'm writing it up as my own Q on the site now. – slm Dec 28 '13 at 5:06
  • @hash - I've added another technique using filefrag. – slm Dec 29 '13 at 10:07
  • @hash - I've added another technique using debugfs. – slm Dec 29 '13 at 17:43
  • i tried filefrag with available blocksizes of 1024 and 2048.. debugfs with a larger file extents: 0 - 12187.. i'll take my time and understand.. that's a great help, thanks... – precise Dec 29 '13 at 18:58
4

Sector number relative to the block device holding the FS (not whole disk)

(Note that hdparm --fibmap is relative to the whole disk, not the partition or whatever other blockdev holds the FS. It also requires root.)

filefrag -e works well, and uses the generic and efficient FIEMAP ioctl, so it should work on pretty much any filesystem (including the often-weird BTRFS, even for BTRFS-compressed files). It will fall back to FIBMAP for filesystems / kernels without FIEMAP support.

$ filefrag xpsp3.vdi          # some old sparse disk image I had lying around
xpsp3.vdi: 110 extents found
$ filefrag -e xpsp3.vdi
Filesystem type is: 58465342
File size of xpsp3.vdi is 5368730112 (1310726 blocks of 4096 bytes)
 ext:     logical_offset:        physical_offset: length:   expected: flags:
   0:        0..       5: 1322629241..1322629246:      6:            
   1:       13..      13: 1322620799..1322620799:      1: 1322629247:
   2:       15..      47: 1323459271..1323459303:     33: 1322620800:
...
 160:   899498..  915839: 1325792977..1325809318:  16342: 1325725438:
 161:  1307294.. 1307391: 1323938199..1323938296:     98: 1325809319: last
xpsp3.vdi: 110 extents found

XFS-only

If you are using xfs, then xfs_bmap has nicer output: It shows you where there are holes, while filefrag just has the next extent starting at a later sector. It uses 512B blocks, not whatever the filesystem blocksize actually is. (typically 4k on Linux). It shows you which allocation-group each extent is in, and how it is aligned on RAID stripe boundaries.

$ xfs_bmap -vvpl xpsp3.vdi   # the extra -v prints a key to the flags
xpsp3.vdi:
 EXT: FILE-OFFSET           BLOCK-RANGE              AG AG-OFFSET              TOTAL FLAGS
   0: [0..47]:              10581033928..10581033975 13 (83912..83959)            48 01111
   1: [48..103]:            hole                                                  56
   2: [104..111]:           10580966392..10580966399 13 (16376..16383)             8 01010
   3: [112..119]:           hole                                                   8
 ...
 322: [10458352..10459135]: 10591505592..10591506375 13 (10555576..10556359)     784 01111
 323: [10459136..10485807]: hole                                               26672
FLAG Values:   # this part is only here with -vv
    010000 Unwritten preallocated extent
    001000 Doesn't begin on stripe unit
    000100 Doesn't end   on stripe unit
    000010 Doesn't begin on stripe width
    000001 Doesn't end   on stripe width

-l is redundant when -v is used, but for some reason I always type -vpl. -pl is more compact output.


Both filefrag and xfs_bmap show you preallocated extents.

$ fallocate --length $((1024*1024*8)) prealloced_file
$ filefrag -e prealloced_file
Filesystem type is: 58465342
File size of prealloced_file is 8388608 (2048 blocks of 4096 bytes)
 ext:     logical_offset:        physical_offset: length:   expected: flags:
   0:        0..    2047: 1325371648..1325373695:   2048:             last,unwritten,eof
prealloced_file: 1 extent found
$ xfs_bmap -vvpl prealloced_file 
prealloced_file:
 EXT: FILE-OFFSET      BLOCK-RANGE              AG AG-OFFSET            TOTAL FLAGS
   0: [0..16383]:      10602973184..10602989567 13 (22023168..22039551) 16384 10010
 FLAG Values:
    010000 Unwritten preallocated extent
    001000 Doesn't begin on stripe unit
    000100 Doesn't end   on stripe unit
    000010 Doesn't begin on stripe width
    000001 Doesn't end   on stripe width
$ dd if=/dev/zero of=prealloced_file conv=notrunc bs=4k count=10 seek=10000
40960 bytes (41 kB) copied, 0.000335111 s, 122 MB/s
$ xfs_bmap -vpl prealloced_file                                           
prealloced_file:
 EXT: FILE-OFFSET      BLOCK-RANGE              AG AG-OFFSET            TOTAL FLAGS
   0: [0..16383]:      10602973184..10602989567 13 (22023168..22039551) 16384 10010
   1: [16384..79999]:  hole                                             63616
   2: [80000..80895]:  10603013120..10603014015 13 (22063104..22063999)   896 00111
 # oops, wrote past EOF and extended the file, instead of in the middle of the preallocated extent
$ dd if=/dev/zero of=prealloced_file conv=notrunc bs=4k count=10 seek=100
40960 bytes (41 kB) copied, 0.000212986 s, 192 MB/s
$ xfs_bmap -vpl prealloced_file 
prealloced_file:
 EXT: FILE-OFFSET      BLOCK-RANGE              AG AG-OFFSET            TOTAL FLAGS
   0: [0..16383]:      10602973184..10602989567 13 (22023168..22039551) 16384 10010
   1: [16384..79999]:  hole                                             63616
   2: [80000..80895]:  10603013120..10603014015 13 (22063104..22063999)   896 00111
# If you check *right away*, XFS's delayed allocation hasn't happened yet.
# FIEMAP on xfs only reflects allocations, which lag behind completed writes.  fsync first if you need it, IIRC.
$ xfs_bmap -vpl prealloced_file 
prealloced_file:
 EXT: FILE-OFFSET      BLOCK-RANGE              AG AG-OFFSET            TOTAL FLAGS
   0: [0..799]:        10602973184..10602973983 13 (22023168..22023967)   800 10111
   1: [800..879]:      10602973984..10602974063 13 (22023968..22024047)    80 01111
   2: [880..16383]:    10602974064..10602989567 13 (22024048..22039551) 15504 11010
   3: [16384..79999]:  hole                                             63616
   4: [80000..80895]:  10603013120..10603014015 13 (22063104..22063999)   896 00111
$ filefrag -e prealloced_file 
Filesystem type is: 58465342
File size of prealloced_file is 41000960 (10010 blocks of 4096 bytes)
 ext:     logical_offset:        physical_offset: length:   expected: flags:
   0:        0..      99: 1325371648..1325371747:    100:             unwritten
   1:      100..     109: 1325371748..1325371757:     10:            
   2:      110..    2047: 1325371758..1325373695:   1938:             unwritten
   3:    10000..   10111: 1325376640..1325376751:    112: 1325373696: last,eof
prealloced_file: 2 extents found

hdparm --fibmap is only useful if you want a sector number relative to the entire hard drive, not within the partition the filesystem is on. It doesn't work on top of software RAID (or presumably anything else between the filesystem and a hard drive). It also requires root. Despite the name of the option, it actually uses FIEMAP when available (the newer extent-map ioctl, not the old slow block-map ioctl).

# hdparm --fibmap ..../xpsp3.vdi
Unable to determine start offset LBA for device, aborting.
0

So, for a given file, you want to know which disk block numbers contain the start and end of that file.

debugfs(8) looks promising for ext2/3/4 FSes

stat(1), ls -i, lsof(8) provide the inode number, but not much else about disk blocks.

head/tail --bytes=1024 is useful for file content, but not disk blocks.

dd(1) will be what you want to inspect the block contents - be alert to the difference between the seek= and skip= parameters, and avoid of=/dev/... unless you really want the output file to be a device.

  • no that's not what I meant... its the disk's block numbers I'm concerned with. – precise Dec 27 '13 at 19:53
0

hdparm --fibmap will list the blocks a file occupies. Note that they may not be contiguous, so "start and end" doesn't make sense.

  • I think the switch you're referring to is --fibmap. Also you need to specify a filename w/ it. Example: hdparm --fibmap afile. – slm Dec 27 '13 at 23:35
  • @slm, oops, yes, typo... and I thought it was obvious you have to name the file in question. – psusi Dec 28 '13 at 1:00
  • It wasn't to me until I tried to run it. Until today my past experience with hdparm was on a whole drive level, never used it for files before. – slm Dec 28 '13 at 1:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.