iostat report huge writes to drives that's not even mounted

Question

root@host [~]# fsck /home2
fsck from util-linux-ng 2.17.2
e2fsck 1.41.12 (17-May-2010)
/dev/sdb1: clean, 6018617/91578368 files, 54524459/366284000 blocks
root@host [~]# fsck /home4
fsck from util-linux-ng 2.17.2
e2fsck 1.41.12 (17-May-2010)
/dev/sdd1: clean, 8094369/91578368 files, 75999625/366284000 blocks

fsck returns no error

root@host [~]# lsof /home4
root@host [~]# lsof /home2

lsof returns no user

root@host [~]# mount
/dev/mapper/VolGroup-lv_root on / type ext4 (rw,relatime,usrjquota=quota.user,jqfmt=vfsv0)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
tmpfs on /dev/shm type tmpfs (rw,rootcontext="system_u:object_r:tmpfs_t:s0")
/dev/sda1 on /boot type ext4 (rw)
/dev/mapper/VolGroup-lv_home on /home type ext4 (rw,relatime,usrjquota=quota.user,jqfmt=vfsv0)
/dev/sdc1 on /home3 type ext3 (rw,relatime)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
/usr/tmpDSK on /tmp type ext3 (rw,noexec,nosuid,loop=/dev/loop0)
/tmp on /var/tmp type none (rw,noexec,nosuid,bind)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
root@host [~]# iostat -xk
Linux 2.6.32-279.19.1.el6.x86_64 (host.buildingsuperteams.com)  01/06/2013      _x86_64_        (16 CPU)

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
          18.91    0.02   39.17   20.22    0.00   21.67

mount shows that there is sdd1 and sdb1 is not mounted

Device:         rrqm/s   wrqm/s     r/s     w/s    rkB/s    wkB/s avgrq-sz avgqu-sz   await  svctm  %util
sda               0.16    11.93    1.35    3.30    53.91    59.92    48.95     0.10   21.87   3.70   1.72
sdb               0.49   219.57   22.00   99.14   224.17  1275.44    24.76     7.44   61.38   7.45  90.24
sdd               0.46   226.39   23.26   92.71   260.61  1277.34    26.52     0.67    5.77   7.71  89.40
sdc               0.00     1.79    0.28    0.05     5.03     7.38    74.28     0.00   14.34   2.05   0.07
dm-0              0.00     0.00    1.45   14.91    53.66    59.50    13.83     1.56   95.36   1.06   1.73
dm-1              0.00     0.00    0.04    0.10     0.18     0.41     8.00     0.00   21.25   2.44   0.04
dm-2              0.00     0.00    0.01    0.00     0.05     0.01     8.49     0.00    7.32   1.84   0.00

iostat report huge writes

What would the reason be? I will replace the hard disk anyway. But this puzzles me to no end

This caused a server crash already. I unmout the drive.

iostat -x 1 shows empty, which is what's expected. So all this time I saw past data?

Answer 1

It is possible your VolGroup-lv_root is created on that drive. Check output of following command

pvs

It display physical volumes information about LV.

More info about LVM (1), (2), (3)