9

Instead of just mounting tmpfs on /var/log I want to use overlayfs.

  1. /var/log are writable tmpfs, but containing files were there before tmpfs mount. This old files are not in memory of tmpfs but in lower layer.
  2. only changes are stored in tmpfs, while old and unmodified files stored on SSD
  3. sometimes it should be possible to write changes to SSD, for example via cron. This should free up tmpfs memory

So, result should be: logs written to RAM, old and new boot logs accesable via same path. Changes are written sometimes to disk, by script.

Point is to speed up a little, and safe SSD from many writes.

(I saw similar thing in puppy linux, not for logs, but for all changes to root, but without installing it can't do the same, documentation not helps)

I will do same for browser cookies/cache based on answer. But persistent write will be done on browser close. Can't turn off browser cache, need at least small cache to have same bugs in my web development as users can have because of cache.

2
  • 1
    "Point is to speed up a little, and save SSD from many writes." All you're really doing is reimplementing the disk cache layer. Commented Jan 19, 2019 at 12:49
  • @roaima still it's an excellent idea for Raspberry Pi , Orange Pi etc. Where a power cut can corrupt your micro SD card turning your system unusable unless you have a protection like an overlay filesystem on some directories.
    – imbr
    Commented Jun 9, 2022 at 19:05

4 Answers 4

6

Managed to make /var/log overlay, it shows SSD log files, and changes. All changes are kept in RAM. Later I'll do syncing, so changes become permanent every hour, by copying upper layer to lower.

#prepare layers
sudo mkdir -p /var/log.tmpfs
sudo mount -t tmpfs -o rw,nosuid,nodev,noexec,relatime,size=512m,mode=0775 tmpfs /var/log.tmpfs
sudo mkdir -p /var/log.tmpfs/upper
sudo mkdir -p /var/log.tmpfs/work
sudo chown -R root:syslog /var/log.tmpfs
sudo chmod -R u=rwX,g=rwX,o=rX /var/log.tmpfs

#prepare overlay
sudo mkdir -p /var/log.overlay
sudo chown root:syslog /var/log.overlay
sudo chmod u=rwX,g=rwX,o=rX /var/log.overlay

#start overlay
sudo mount -t overlay -o rw,lowerdir=/var/log,upperdir=/var/log.tmpfs/upper,workdir=/var/log.tmpfs/work overlay /var/log.overlay
sudo mount --bind /var/log.overlay /var/log

To make changes persistent, its needed to unmount bind /var/log, copy files, then bind again.

9
  • 1
    why? what actual benefit do you gain from this? quickly fill up your RAM to trigger the kernel's OOM random-process-killer faster?
    – cas
    Commented Feb 16, 2018 at 4:54
  • 3
    Max size of tmpfs is set, so ram will not fill up. logrotate configs are tuned the way it never fill all reserved size. Anyway, I find out its not possible to make changes persistent online, so I have to deceide either remove this script or loose logs on reboot. This is normal for development server, not production.
    – LeonidMew
    Commented Feb 16, 2018 at 7:58
  • 1
    I still don't get what you were trying to achieve. Where's the benefit? there's certainly no performance benefit. It just seems like extra complication and fragility for no good reason. Probably qualifies as some form of premature optimisation: maybe something like "ram disks are fast so i'll put my logs on a tmpfs" without testing whether there's any actual, noticeable benefit. Hint: unless the logs are fsynced on every write, writes will be buffered anyway.
    – cas
    Commented Feb 16, 2018 at 8:02
  • 6
    This is an excellent solution for a Raspberry Pi that loses power often and is subject to disk corruption if the SD card is writing while power is lost. Several key directories must be overlaid in addition to /var/log, but this solution lays the groundwork for creating a stable, read-only with ephemeral read/write support.
    – Brad Hein
    Commented Dec 29, 2019 at 19:36
  • 3
    The solution might not be perfect but the thinking behind is brilliant
    – Daniel W.
    Commented Sep 7, 2020 at 20:23
6

While the accepted answer by LeonidMew is almost perfect, I would like to contribute a way to persist changes without the need to remount.

# prepare layers
sudo mkdir -p /var/log.tmpfs
sudo mount -t tmpfs -o rw,nosuid,nodev,noexec,relatime,size=512m,mode=0775 tmpfs /var/log.tmpfs
sudo mkdir -p /var/log.tmpfs/upper
sudo mkdir -p /var/log.tmpfs/work
sudo chown -R root:syslog /var/log.tmpfs
sudo chmod -R u=rwX,g=rwX,o=rX /var/log.tmpfs

# create a bind mount to the parent directory of the log folder
# since mount --bind does not mount recursively when used on the
# parent, we can use this to preserve access to the real /var/log
# on the disk
sudo mkdir -p /tmp/realvar
sudo mount --bind /var /tmp/realvar

# mount overlay using this new mountpoint as lowerdir
sudo mount -t overlay -o rw,lowerdir=/tmp/realvar/log,upperdir=/var/log.tmpfs/upper,workdir=/var/log.tmpfs/work overlay /var/log

With this method /tmp/realvar/log is used as the lowerdir and remains accessible even while the overlay is mounted. We can use this mountpoint to synchronize changes:

sudo rsync -vaz --delete /var/log/ /tmp/realvar/log

Thanks LeonidMew for the great idea. This concept is very useful for Raspberry Pis to reduce the write cycles on the SD card while still preserving most of the logs in case of an accidental power loss.

2
  • Excelent! This helped understand that mount on a non empty folder 'hides' its contents and the bind is just 'unhiding' it somewhere else to make it usable as lowerdir.
    – imbr
    Commented Jun 10, 2022 at 0:37
  • Nice one, but can we do the same to remount / as overlayfs, because you cannot create a realroot relative to / as its going to be remounted, is there a way to refer to previous mount in that case? Commented Sep 10, 2022 at 10:08
3

I can suggest anything-sync-daemon. It pretty good job.

https://wiki.archlinux.org/index.php/anything-sync-daemon

From the documentation,

anything-sync-daemon (asd) is a tiny pseudo-daemon designed to manage user specified directories referred to as sync targets from here on out, in tmpfs and to periodically sync them back to the physical disc (HDD/SSD). This is accomplished via a symlinking step and an innovative use of rsync to maintain synchronization between a tmpfs copy and media-bound backups. Additionally, asd features several crash recovery features.

0

To minimize flash drive writes, I'm using log2ram, and it works really well. It syncs to the flash drive on a schedule I supply 👍

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .