What occupies space in overlayfs

Question

I have a linux with a read only root filesystem and a read-write overlayfs mounted over it:

# mount
overlayfs on / type overlayfs (rw,relatime,lowerdir=/root_ro/,upperdir=/root_rw/)
...

The overlayfs is almost full

# df
Filesystem           1K-blocks      Used Available Use% Mounted on
overlayfs              4003548   3995012      8536  99% /
...

How can I identify files consuming the read/write part of the overlayfs? The du does not differentiate space occupied on ro and rw media. I have found the option -fstype type in find but my linux has busybox and the find does not support this option there.

EDIT: add output from cat /proc/mounts

rootfs / rootfs rw 0 0
proc /proc proc rw,relatime 0 0
sysfs /sys sysfs rw,relatime 0 0
none /dev devtmpfs rw,relatime,size=1026976,nr_inodes=256744,mode=755 0 0
/dev/sda1 /root_rw ext4 rw,relatime,errors=remount-ro,data=ordered 0 0
ubi0:rootfs /root_ro ubifs ro,noatime,nodiratime 0 0
overlayfs / overlayfs rw,relatime,lowerdir=/root_ro/,upperdir=/root_rw/ 0 0
debugfs /sys/kernel/debug debugfs rw,relatime 0 0
devpts /dev/pts devpts rw,relatime,gid=5,mode=620 0 0

Answer 1

There isn't really a notion of “what occupies the space” in an overlay filesystem. Each branch of the union has its own space occupation. Run du on both branches. If it's getting more full, the read-write branch is the culprit.

Since the overlay mount shadows its branches (/root_ro and /root_rw are hidden by the mount on /), you need to gain access to the branches. You can do that by mounting the block device again (Linux supports this, at least for most block device types):

mkdir /media/root_ro /media/root_rw
mount /dev/sda1 /mnt/root_rw
mount ubi0:rootfs /mnt/root_ro
du /mnt/root_ro /mnt/root_rw

Answer 2

Unmount the overlay filesystem, and then mount it somewhere else and check it using du. If I understand them correctly, that should let you see what's in it.