I have usb keychain with size of 7.5GB and I need to copy file on it with size 7.4GB. But I can't because superblocks consume 0.5GB of space.

  • 1
    Does it have to be ext4 specifically? I'd probably try ext2: something like mke2fs -t ext2 -N 64 -O sparse_super. That said, I'm not sure you can get the filesystem overhead down quite as far as you'd need (to less than 1.3% of the device capacity). – a CVn Oct 6 '12 at 18:56
  • I don't care about filesystem if it's able to handle files with size bigger that 4GB. – kravemir Oct 6 '12 at 18:58
  • 3
    Do you need a file system? What about putting all the files in a tar archive and write that archive directly on the block device? – Marco Oct 6 '12 at 20:15
  • @Marco That's an interesting approach that is crazy enough that it just might work. You might want to make an answer out of that one, since it clearly addresses the OP's problem statement even though not the literal question. – a CVn Oct 6 '12 at 21:55
  • @MichaelKjörling "That's an interesting approach that is crazy enough that it just might work." Nothing crazy about it. It used to be very common when saving stuff to tape. – dmckee Oct 6 '12 at 22:03

You don't need a file system to write data to a device. You can simply use tar to create an archive that stores your directory structure and all meta data and write that to the device.

Writing data

Here sdb is an example of the USB drive on my system, adjust according to your setup.

tar cf /dev/sdb <some_directory>

Reading data

You can directly use tar to read the data from the device:

tar xf /dev/sdb

In my experiments this always reads the entire block device, not just the data in the tar archive. If you know that your device has 8 GiB but you only saved, say 3 GiB, you can use dd to avoid reading the entire device:

dd if=/dev/sdb bs=1M count=3072 | tar xf -

Side notes

Try to compress the data as much as possible. This might take a long time, but maybe everything fits on a drive with an ordinary filesystem. I would advice to use 7-Zip, it's slow but it has a high compression ratio. Here is an example:

7za a -t7z -m0=lzma -mx=9 -mfb=64 -md=32m -ms=on archive.7z <some_directory>
  • 1
    It's compressed HD video, so another compression would be only waste of time. How can I write it without compression? – kravemir Oct 7 '12 at 11:04
  • 1
    Just use tar like described in “Writing data” and “Reading data” and skip what I've written in “Side notes”. Tar does not compress by default. – Marco Oct 7 '12 at 11:12
  • I'm gonna try it today. If it works i'll accept your question :) – kravemir Oct 7 '12 at 11:55

Since you don't really need an ext4 file system but are really asking about a file system that will bring the overhead down below about 1.3% of the device capacity (100 MB out of 7.5 GB), I'd look at various low-overhead options. The two most obvious that meet your criteria of being able to handle a single 7.4 GB file is either ext2 with a low inode count and sparse superblocks, or a low-overhead FAT32 file system.

EDIT: It looks like I was wrong about FAT32's maximum file size, but I'm leaving it here in case someone comes across this and can live with the limitation that a single file cannot be larger than 2^32 - 1 (4 GiB - 1) bytes.

For a low-overhead ext2 file system, try something along the lines of mke2fs -t ext2 -N 8 -O sparse_super. The -N 8 specifies the number of inodes on the file system. I don't know how low this will go, so this knob may require a bit of twiddling.

For a low-overhead FAT32 file system, try something like mkdosfs -F 32 -f 1 -r 8 -S 32768 -s 128 -a. This will create a -F 32 FAT32 (which allows the file size you need), with -f 1 a single FAT, -r 8 8 root directory entries, -S 32768 a logical sector size of 32 KiB, -s 128 128 sectors per cluster (you might be able to increase this even further) and -a disable data structure alignment.

  • 1
    FAT32 won't work (he needs to copy files > 4 GB). – Renan Oct 6 '12 at 19:36
  • 1
    @Renan Good catch. I thought that increasing the logical sector size would get around that limitation, but it looks like it doesn't because of the 32-bit file byte size field. – a CVn Oct 6 '12 at 19:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.