The second thing I looked for was direct device access.
sudo lsof /dev/sd* showed nothing.
cd /dev; fatrace --current-mount --timestamp didn't show any associated accesses either.
At this point I needed to start stripping down to the kernel. Lets try
systemctl isolate rescue.target. Strange, it bounces back to
systemctl status shows the system is degraded because
dmeventd didn't want to stop while it's still monitoring devices (!)... but the hard drive has stopped spinning back up (!!).
Indeed, on a non-degraded system, the problem goes away after
killall -9 dmeventd.
How can it be so broken? The reason is I started playing with
docker, and since I use LVM it chose the devicemapper storage driver.[*]
dmeventd: dmeventd ready for processing.
lvm: Monitoring thin vg_fossil-docker--pool.
[*] It might also be a problem if you have LVM mirrors, raid, or snapshots... admittedly a possibility on a NAS system :(. If you don't have any of these, e.g. just simple LVs or no LVM at all, then dmeventd doesn't have anything to monitor and it behaves itself.
Specifically dmeventd is doing these ioctls about every 10 seconds:
open("/dev/mapper/control", O_RDWR) = 7
I still think the spinup is a bug, but at least for my case there's an
obvious workaround[*] and I'm not really worried I'm going to miss anything because of it.
[*] If you really want to stop dmeventd working, even though you have a thin pool which probably dies horribly if it runs out of space, search for
monitoring = 1 in
lvm.conf and change the value to
Update: The bug is fixed in the very next version of