Writing and deleting many files quickly
I'm going to write many temporary files per second, over a sustained period. Each will be deleted after some milliseconds.
Say we have an image processing software, that works on partial frames from a fast camera, about 2000 per second. Each of them is written to a file, then the file is changed a little, read again, and deleted, in the same couple of milliseconds.
This currently runs on a tmpfs
filesystem, but let's assume it can not.
And no, I can not "just fix it doing that", it's needed for some legacy integration.
How's the filesystem coping?
I would like to understand how this is interacting with the filesystem options. Specifically, I can not get my head wrapped around how a filesystem journal interacts with this.
As a naiive idea, one could hope the data does never touch the disk, and even the metadata does not; After creating a file and then deleting it, the data is the same, right? And after writing and deleting the name in the directory, the directory is also the same, right?
Surprising effects
No, wrong.
A directory can represent the same file names in multiple ways, like by reusing one place to store a new file name, or another place.
That means, when there where write operations on a disk block, it may have changed without filesystem level changes, so it needs to be written to the disk.
But can I make sure it's not written once for each of the four operations? And what about the journal? Will my temporary data end up in the journal? Can I prevent that?
As one question: what should I take care of or keep in mind when doing that?