Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I have a sparse file, in which only some blocks are allocated:

~% du -h --apparent-size example
100K    example
~% du -h example
52K     example

I would like to know which blocks of the file are actually allocated. Is there a system call or kernel interface that could be used to get a list of either the allocations, or the holes of file?

Simply checking for a long enough string of zeros (the approach used by GNU cp, rsync, etc) does not work correctly:

~% cp example example1  
~% du -h example1 
32K     example1

It detected other sequences of zeros that were actually allocated.

share|improve this question

2 Answers 2

up vote 4 down vote accepted

There is a similar question on SO. The currently accepted answer by @ephemient suggests using an ioctl called fiemap which is documented in linux/Documentation/filesystems/fiemap.txt. Quoting from that file:

The fiemap ioctl is an efficient method for userspace to get file extent mappings. Instead of block-by-block mapping (such as bmap), fiemap returns a list of extents.

Sounds like this is the kind of information you're looking for. Support by filesystems is again optional:

File systems wishing to support fiemap must implement a ->fiemap callback on their inode_operations structure.

Support for the SEEK_DATA and SEEK_HOLE arguments to lseek you mentioned from Solaris was added in Linux 3.1 according to the man page, so you might use that as well. The fiemap ioctl appears to be older, so it might be more portable across different Linux versions for now, whereas lseek might be more portable across operating systems if Solaris has the same.

share|improve this answer
1  
You can get this FIEMAP information by using the ̀--fibmap of the hdparm utility. See the manual. –  Totor Mar 3 '13 at 20:51

It depends on the file system. I don't believe their is a call, which may be why many tools don't handle copying sparse files well. The GNU tool chain use searching for large blocks of zeros as that allows them to remove unused allocated blocks. Many copy tools will convert a sparse file into a file with all blocks allocated.

You will likely have to open the inode, and parse the result. Inode format is file system dependent. Some file systems may have part of your data in the inode itself.

share|improve this answer
1  
There has to be some FS-agnostic way to have this information. Reading directly from the inode is definitely not an option. I was looking for something like SEEK_DATA and SEEK_HOLE parameters for lseek(), like there are in Solaris: opensolarisforum.org/man/man2/lseek.html –  Juliano Feb 6 '11 at 22:04
    
@Juliano A look at the Linux lseek option doesn't have these options. Solaris supports very few file systems, so it would be relatively easy to support. Linux supports a wide variety of file systems, some of which do not support sparse files. Support for SEEK_DATA / SEEK_HOLE would impose support in code for all the file systems. These methods may not do what you expect. See blogs.sun.com/bonwick/entry/seek_hole_and_seek_data for more data from the Sun side. –  BillThor Feb 6 '11 at 22:20
1  
Filesystems don't need to support anything with the lseek() interface, the kernel witelists the filesystem modules that support SEEK_DATA/SEEK_HOLE through a module property. This is in the manpage itself and the linked blog: "For filesystems that do not supply information about holes, the file will be represented as one entire data region." –  Juliano Feb 6 '11 at 22:38
    
@Juliano Still requires kernel mods as well as changes to lseek. As per the blog entry this is fairly new functionality at Sun. For it to work the file system code needs to modified as well. It certainly would require changes to all the file systems supporting sparse files to provide the kernel hooks. –  BillThor Feb 6 '11 at 22:46

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.