2

I'm working with memory mapped files. Each file is allocated using truncate (see man 2 truncate). These are then memory mapped and the application writes entries into various sections of the files as it requires. This part is easy.

I have a problem with how to delete the entries from disk as they are no-longer needed. This is to save disk space not any GDPR "must be deleted". Entries don't necessarily expire with time, but with lack of use. So while the files are generated in the order the entries are created, they cannot be deleted in the same order.

I see three options:

  1. Investigate deleting sections of a sparse file, making them more sparse (This question!).
  2. Leave expired entries on disk until every entry in the file has expired. The worst case here is several TB of extra data on disk.
  3. Come up with a mechanism to copy "tenured" entries (old but still in-use). This is much more complex than it sounds as it requires handling millions of cross references.

Is there a way in Linux to delete sections of a sparse file or is this mechanism opaque to applications?

1 Answer 1

2

Under Linux, you can punch holes in a file using fallocate()’s FALLOC_FL_PUNCH_HOLE flag (combined with FALLOC_FL_KEEP_SIZE):

fallocate(fd, FALLOC_FL_PUNCH_HOLE | FALLOC_FL_KEEP_SIZE, offset, length);

On file systems which support this operation, this will increase the file’s sparseness if the offset and length combination cover entire blocks; anything else will only be zeroed. On file systems which don’t support this operation, an error will be returned.

See also What are functions to manupulate sparse files under Linux?

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .