10

To my understanding, for manipulating files there is only the sys_write syscall in Linux, which overwrites the file content (or extends it, if at the end).

Why are there no syscalls for inserting or deleting content in files in Linux?

As all current file systems do not require the file to be stored in a continuous memory block, an efficient implementation should be possible. (The files would get fragmented.)

With file system features as "copy on write" or "transparent file compression", the current way of inserting content seems to be very inefficient.

  • 4
    As with all fancy file operations, such an operation is much less useful in practice than it appears. The main use for such a thing are very specialized applications, like databases, emulators and such. The way you usually "edit" a file is by creating a newfile and and have a "save" operation by the user rename the new file to the old. – mosvy Jun 16 at 13:33
  • 3
    @mosvy, but is the "create new file, then rename" concept used because it's good in itself, or exactly because the system doesn't provide any better way? Especially on text files operations like "modify this line (changing the length)" or "insert these lines here" are rather common, so one could assume that filesystem operations for those exact functions would be used if they were there. Of course, not having them makes the fs implementation much simpler... – ilkkachu Jun 16 at 16:01
  • 1
    @meuh OpenVMS still does, via RMS (Record Management Services). – RonJohn Jun 16 at 21:05
  • 1
    UNIX started a move away from providing record management systems inside the file system. – user207421 Jun 17 at 1:33
  • 1
    @ilkkachu it's good in itself, absolutely no doubt ;-) Even more, If inodes were immutable, that will make implementing block sharing, versioning, and almost everything much more efficient (and much more simple to reason about). Think by analogy how all script languages have switched to immutable strings -- but I'll cut it short here; it's hard to talk off the cuff about filesystems and not sound like a quack ;-) – mosvy Jun 17 at 16:18
22

On recent Linux systems that is actually possible, but with block (4096 most of the time), not byte granularity, and only on some filesystems (ext4 and xfs).

Quoting from the fallocate(2) manpage:

int fallocate(int fd, int mode, off_t offset, off_t len);

[...]

Collapsing file space

Specifying the FALLOC_FL_COLLAPSE_RANGE flag (available since Linux 3.15) in mode removes a byte range from a file, without leaving a hole. The byte range to be collapsed starts at offset and continues for len bytes. At the completion of the operation, the contents of the file starting at the location offset+len will be appended at the location offset, and the file will be len bytes smaller.

[...]

Increasing file space

Specifying the FALLOC_FL_INSERT_RANGE flag (available since Linux 4.1) in mode increases the file space by inserting a hole within the file size without overwriting any existing data. The hole will start at offset and continue for len bytes. When inserting the hole inside file, the contents of the file starting at offset will be shifted upward (i.e., to a higher file offset) by len bytes. Inserting a hole inside a file increases the file size by len bytes.

  • 1
    "but with block (4096), not byte granularity" - 4KiB blocks are very common in ext4, but that's not guaranteed. Ext4 supports 1KiB, 2KiB and 4KiB block sizes; and I remember from the ext2 days that on Alpha processors, 8KiB was supported as well. You can't just assume blocks are 4KiB, I'm afraid. – marcelm Jun 17 at 15:20
  • 1
    4k (which is the default) is a multiple of 1k and 2k, so there's no problem with assuming 4k with ext4. While xfs will default to 4k too, it is supposed to support a bs larger than 4k -- up to 64k, but I was only able to create such a fs -- mounting it fails with no ENOSYS. And anyways, you cannot assume anything -- this feature is not supported on all fs, so it's better to just say block = 4096, so the reader have some sense of proportion, rather than letting it float and letting people it could be anything, or worse, that it's 512 bytes or is somehow related to the vm page size. – mosvy Jun 17 at 16:12
  • After you edit (where you say it's usually 4KiB), I fully agree! My problem was that previously it was easily read as "blocks are always 4KiB", which may cause people to make that assumption and write buggy code. – marcelm Jun 17 at 17:00
9

As all current file systems do not require the file to be stored in a continuous memory block,

Filesystems might not require files to be stored in a continuous area (and that would be very inflexible indeed), but usually files are stored in fixed-size blocks (or sequences of contiguous blocks). Doing it that way simplifies the implementation, and the blocks are usually multiples of the block size of the underlying device.

So, implementing inserts of blocks with arbitrary length would make the file system format and implementation rather more complex or require moving potentially large amounts of data around. Neither of those is really good, and complex data structures can be built in userspace on top of the filesystem API.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.