How does SquashFS handle decompression?

Question

How does SquashFS handle decompression? I'm interested in the overhead that LZ4 has on the SquashFS filesystem. My theory is that the files in the SquashFS filesystem gets uncompressed during read. What about directory listing?

Answer 1

SquashFS is documented in some detail in the kernel documentation. All the contents of a SquashFS file system can be compressed, including inodes and directory entries; inodes and directory entries are also laid out specifically to improve compression.

This means that any data or metadata read from the file system involves decompression; however, the overhead is reduced somewhat using caches:

• file data is decompressed to the page cache, so multiple reads in the same page don’t require decompressing the data multiple times;
• metadata and fragments (tail-end packed blocks) are decompressed to small, specific metadata and fragment caches, so multiple reads of the same metadata blocks (e.g. when listing a directory’s contents) don’t require decompressing the same blocks multiple times.

There are also various lookup tables to speed up operations involving various indexes (blocks, user and group identifiers, and fragments).

Answer 2

From man mksquashfs (emphasis mine):

quashfs is a highly compressed read-only filesystem for Linux. It uses either gzip/xz/lzo/lz4/zstd compression to compress both files, inodes and directories.

So, it also compresses directory contents.

By the way, you can control whether the inode, uid/gid tables, data or data fragment blocks and extended attributes are compressed using the -no{I,Id,D,F,X} flags.

Re: overhead: you'll find that on many systems, decompression is much faster than reading from permanent storage, so that the overhead in terms of speed is actually negative.