0

I have a process on a Linux machine that fetches some chunk of data from a host and sends it to another host. I do not control the size of the chunks I fetch, but I have a general idea of their median/average size.

I don't want to pay the cost of writing the data to the disk, as most of the time the chunks fit in RAM, so it would be a waste to write them on disk to reread them and delete them just after that.

What I would want to do is have a filesystem like tmpfs, with a limit on the in-RAM size, that swaps to disk when the limit is reached.

I tried to use a tmpfs with a lot of swap, but Linux tends to swap my programs instead of swapping the content of the tmpfs, and it deadlocks the machine.

What could I use to avoid paying the cost of disk writes for files I'm going to delete soon?

2
  • Raise commit= mount option. Commented Nov 12, 2018 at 12:20
  • @IporSircer ... if you are using ext3/ext4. But it sounds like you can still get unlucky and be hit by a commit anyway, it just decreases the frequency? Maybe XFS is better. And if you want more than 30 seconds for any FS, you must also raise the sysctl vm.dirty_expire_centiseconds.
    – sourcejedi
    Commented Nov 12, 2018 at 12:54

1 Answer 1

2

Normal file cashing should do this for you without any extra work. Or use the commit mount option. Or use lvmcache. Or use a pipe.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .