How to create low-overhead file system

Question

I have usb keychain with size of 7.5GB and I need to copy file on it with size 7.4GB. But I can't because superblocks consume 0.5GB of space.

Answer 1

You don't need a file system to write data to a device. You can simply use tar to create an archive that stores your directory structure and all meta data and write that to the device.

Writing data

Here sdb is an example of the USB drive on my system, adjust according to your setup.

tar cf /dev/sdb <some_directory>

Reading data

You can directly use tar to read the data from the device:

tar xf /dev/sdb

In my experiments this always reads the entire block device, not just the data in the tar archive. If you know that your device has 8 GiB but you only saved, say 3 GiB, you can use dd to avoid reading the entire device:

dd if=/dev/sdb bs=1M count=3072 | tar xf -

Side notes

Try to compress the data as much as possible. This might take a long time, but maybe everything fits on a drive with an ordinary filesystem. I would advice to use 7-Zip, it's slow but it has a high compression ratio. Here is an example:

7za a -t7z -m0=lzma -mx=9 -mfb=64 -md=32m -ms=on archive.7z <some_directory>

Answer 2

Since you don't really need an ext4 file system but are really asking about a file system that will bring the overhead down below about 1.3% of the device capacity (100 MB out of 7.5 GB), I'd look at various low-overhead options. The two most obvious that meet your criteria of being able to handle a single 7.4 GB file is either ext2 with a low inode count and sparse superblocks, or a low-overhead FAT32 file system.

EDIT: It looks like I was wrong about FAT32's maximum file size, but I'm leaving it here in case someone comes across this and can live with the limitation that a single file cannot be larger than 2^32 - 1 (4 GiB - 1) bytes.

For a low-overhead ext2 file system, try something along the lines of mke2fs -t ext2 -N 8 -O sparse_super. The -N 8 specifies the number of inodes on the file system. I don't know how low this will go, so this knob may require a bit of twiddling.

For a low-overhead FAT32 file system, try something like mkdosfs -F 32 -f 1 -r 8 -S 32768 -s 128 -a. This will create a -F 32 FAT32 (which allows the file size you need), with -f 1 a single FAT, -r 8 8 root directory entries, -S 32768 a logical sector size of 32 KiB, -s 128 128 sectors per cluster (you might be able to increase this even further) and -a disable data structure alignment.