hard disks spin up by processes / applications that simply get a list of disks? How to prevent?

Question

I have a 24/7 always-on Debian Jessie based headless home server that has a large 1TB SSD for the OS and all of my frequently accessed files. This same system has 4 larger hard disk drives in a SnapRAID array. These are mainly for archiving infrequently accessed Blu-rays and want those drives to remain spun down in standby unless I actually read or write to them. They are all formatted as ext4 and mounted with noatime and nodiratime enabled.

So even though no process or program should be regularly accessing those drives in any direct way, the hard drives constantly get spun up from standby. It seems to be related to graphical programs that provide a gui file browser, even something like Chromium. If I don't even browse into those drives, I'm thinking that these processes by simply getting a list of available drives spins up the hard disks. Much like blkid does. The problem is, it's hard to determine the root cause of this since none of these processes are actually reading or writing the filesystem on those drives, so no files are actually changing or being touched. Is there some sort of cache that I can populate or a buffer to prevent these programs from spinning up the hard drive simply by getting a list of available disks? This is honestly driving me insane, since I can't find a reliably way to keep these disks spun-down even though there is no direct access of the filesystem.

UPDATE: Thanks to Stephen's answer, I was able to trace the disk activity to gvfs and udisks. It's a real shame that these processes insist on waking up disks in standby when they aren't actually being accessed to do any real I/O with the filesystem. So far I just uninstalled them, knowing that it will remove some functionality from PCManFM and the like.

Answer 1

You can use blktrace (available in Debian) to trace all the activity to a given device; for example

sudo blktrace -d /dev/sda -o - | blkparse -i -

or just

sudo btrace /dev/sda

will show all the activity on /dev/sda. The output looks like

  8,0    3       51   135.424002054 16857  D  WM 167775248 + 8 [kworker/u16:0]
  8,0    3       52   135.424011323 16857  I  WM 209718336 + 8 [kworker/u16:0]
  8,0    3        0   135.424011659     0  m   N cfq496A  / insert_request

The fifth column is the process identifier, and the last one gives the process name when there is one.

You can also store traces for later analysis; blktrace includes a number of analysis tools such as the afore-mentioned blkparse and btt. blktrace is a very low-level tool so it may not be all that easy to figure out what caused activity in the first place, but with the help of the included documentation (see /usr/share/doc/blktrace if you installed the Debian package) and the blktrace paper it should be possible to figure out what's causing the spin-ups.

Answer 2

Do try strace.

strace -f -e open ls

The above command should list out all disk activity related to open methods, which you suspect are responsible for causing your disk spinups.

Alternatively, dmesg might give you some information on physical disk mounts/unmounts.

This site has a pretty good guide on using strace to monitor disk activity.

As for populating a cache to stop disk IO I doubt that's possible, but I may be wrong.