Mount /var/logs as tmpfs, with help of overlayfs to save changes sometimes

Question

Instead of just mounting tmpfs on /var/log I want to use overlayfs.

1. /var/log are writable tmpfs, but containing files were there before tmpfs mount. This old files are not in memory of tmpfs but in lower layer.
2. only changes are stored in tmpfs, while old and unmodified files stored on SSD
3. sometimes it should be possible to write changes to SSD, for example via cron. This should free up tmpfs memory

So, result should be: logs written to RAM, old and new boot logs accesable via same path. Changes are written sometimes to disk, by script.

Point is to speed up a little, and safe SSD from many writes.

(I saw similar thing in puppy linux, not for logs, but for all changes to root, but without installing it can't do the same, documentation not helps)

I will do same for browser cookies/cache based on answer. But persistent write will be done on browser close. Can't turn off browser cache, need at least small cache to have same bugs in my web development as users can have because of cache.

Answer 1

Managed to make /var/log overlay, it shows SSD log files, and changes. All changes are kept in RAM. Later I'll do syncing, so changes become permanent every hour, by copying upper layer to lower.

#prepare layers
sudo mkdir -p /var/log.tmpfs
sudo mount -t tmpfs -o rw,nosuid,nodev,noexec,relatime,size=512m,mode=0775 tmpfs /var/log.tmpfs
sudo mkdir -p /var/log.tmpfs/upper
sudo mkdir -p /var/log.tmpfs/work
sudo chown -R root:syslog /var/log.tmpfs
sudo chmod -R u=rwX,g=rwX,o=rX /var/log.tmpfs

#prepare overlay
sudo mkdir -p /var/log.overlay
sudo chown root:syslog /var/log.overlay
sudo chmod u=rwX,g=rwX,o=rX /var/log.overlay

#start overlay
sudo mount -t overlay -o rw,lowerdir=/var/log,upperdir=/var/log.tmpfs/upper,workdir=/var/log.tmpfs/work overlay /var/log.overlay
sudo mount --bind /var/log.overlay /var/log

To make changes persistent, its needed to unmount bind /var/log, copy files, then bind again.

Answer 2

While the accepted answer by LeonidMew is almost perfect, I would like to contribute a way to persist changes without the need to remount.

# prepare layers
sudo mkdir -p /var/log.tmpfs
sudo mount -t tmpfs -o rw,nosuid,nodev,noexec,relatime,size=512m,mode=0775 tmpfs /var/log.tmpfs
sudo mkdir -p /var/log.tmpfs/upper
sudo mkdir -p /var/log.tmpfs/work
sudo chown -R root:syslog /var/log.tmpfs
sudo chmod -R u=rwX,g=rwX,o=rX /var/log.tmpfs

# create a bind mount to the parent directory of the log folder
# since mount --bind does not mount recursively when used on the
# parent, we can use this to preserve access to the real /var/log
# on the disk
sudo mkdir -p /tmp/realvar
sudo mount --bind /var /tmp/realvar

# mount overlay using this new mountpoint as lowerdir
sudo mount -t overlay -o rw,lowerdir=/tmp/realvar/log,upperdir=/var/log.tmpfs/upper,workdir=/var/log.tmpfs/work overlay /var/log

With this method /tmp/realvar/log is used as the lowerdir and remains accessible even while the overlay is mounted. We can use this mountpoint to synchronize changes:

sudo rsync -vaz --delete /var/log/ /tmp/realvar/log

Thanks LeonidMew for the great idea. This concept is very useful for Raspberry Pis to reduce the write cycles on the SD card while still preserving most of the logs in case of an accidental power loss.

Answer 3

I can suggest anything-sync-daemon. It pretty good job.

https://wiki.archlinux.org/index.php/anything-sync-daemon

From the documentation,

anything-sync-daemon (asd) is a tiny pseudo-daemon designed to manage user specified directories referred to as sync targets from here on out, in tmpfs and to periodically sync them back to the physical disc (HDD/SSD). This is accomplished via a symlinking step and an innovative use of rsync to maintain synchronization between a tmpfs copy and media-bound backups. Additionally, asd features several crash recovery features.

Answer 4

To minimize flash drive writes, I'm using log2ram, and it works really well. It syncs to the flash drive on a schedule I supply 👍