0

Is it possible to completely disable sparse file support in an ext4 file system?

The purpose is to avoid disk fragmentation.

Bonus "no-points" if the solution allows the file system to allocate the files quickly, without filling the file with zeroes (or whatever).

What I want is to be able to tell applications to create files of X bytes and, no matter what method they use to pre-allocate the files, a file of X bytes will be created on the disk. I have no use for an "apparent size" of X bytes, nor I have control over those applications' source codes.

In order to save you some time, let me already address some comments that always seem to appear whenever someone mentions sparse files and quick allocations

  • I don't care how good a file system allocation heuristic is supposed to be.
  • I am aware of the security implications of not zeroing newly allocated blocks.
0
1

There’s no option to disable sparse file support in ext4, but ext4 (and the Linux kernel in general) support features which would allow you to implement a workaround without writing a new file system driver (or adapting the ext4 driver).

I think what could work is to implement a shared library shim, which you would load using LD_PRELOAD, and you’d override all the calls which can result in the creation of sparse files to handle them in such a way that they don’t. The fast way to allocate blocks in files is to use posix_fallocate, which ensures that disk space is allocated when it returns, without necessarily writing all zeroes to the blocks (but the file system guarantees that reading from the allocated blocks will return zeroes). Your shim would have to intercept posix_fallocate too, since it can also be used to create sparse files...

There are limitations to LD_PRELOAD shims (in particular with setuid binaries), but they might not apply in your case.

3
  • Nice idea, but having something like that would be a maintenance nightmare. I was hoping there was some built-in solution I could enable using some obscure mount option. Heck, even a re-format was not out of the picture. Too bad. Anyway, do you know any other file system that would allow me to do something like that? – user1593842 Apr 23 '20 at 19:49
  • Non-POSIX file systems might not support sparse files, so depending on your other requirements, they may fit the bill; FAT or ExFAT in particular would be good candidates. – Stephen Kitt Apr 28 '20 at 18:27
  • Yeah... I'm currently trying to choose between that and getting a bucket load of RAM and then transfer all the data back to disk when I'm done with it. – user1593842 May 3 '20 at 12:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.