Writing and deleting many files quickly
I'm going to write many temporary files per second, over a sustained period. Each will be deleted after some milliseconds.
Say we have an image processing software, that works on partial frames from a fast camera, about 2000 per second. Each of them is written to a file, then the file is changed a little, read again, and deleted, in the same couple of milliseconds.
This currently runs on a
tmpfs filesystem, but let's assume it can not.
And no, I can not "just fix it doing that", it's needed for some legacy integration.
How's the filesystem coping?
I would like to understand how this is interacting with the filesystem options. Specifically, I can not get my head wrapped around how a filesystem journal interacts with this.
As a naiive idea, one could hope the data does never touch the disk, and even the metadata does not; After creating a file and then deleting it, the data is the same, right? And after writing and deleting the name in the directory, the directory is also the same, right?
A directory can represent the same file names in multiple ways, like by reusing one place to store a new file name, or another place.
That means, when there where write operations on a disk block, it may have changed without filesystem level changes, so it needs to be written to the disk.
But can I make sure it's not written once for each of the four operations? And what about the journal? Will my temporary data end up in the journal? Can I prevent that?
As one question: what should I take care of or keep in mind when doing that?