4

I am trying to increase the swap space available to my server from 4GB to 16GB, however at present XFS + something is making this impossible.

I have a single SSD hosting the OS (CentOS 7), with a boot partition and a LVM2 partition, the defaults of the installation. Inside the LVM2 partition are three virtual partitions, root, home, and swap, consuming the entire LVM2 partition. Again by the defaults of the installation, these are in the XFS filesystem type.

My simple want was to shrink the under-utilised home partition, making space for the swap partition. Having now discovered that shrinking XFS partitions is impossible, I am down the rabbit hole of backing up the home drive deleting the partition and reinstating it in smaller size with a recover from the backup.

The problem now is I cannot seem to free the home directory or the partition. Having dropped out of the GUI, logged out and back in as root at the terminal, initially I could not unmount the home directory. Something was in use though lsof and fuser showed nothing open on it.

I issued a

umount -f home

and this appears to have worked in unmounting home. I can now freely mount and unmount home without the -f callout, which seems suspicious. However now trying

lvremove /dev/centos/home

the message returned is

Logical volume centos/home in use.

I have looked at many questions and answers here and it is none of the following:

  • NFS service
  • Open files
  • Open files using the major,minor designation
  • the Active flag (have tried lvchange -an -v /dev/centos/home, it also claimed the volume was in use)

Nothing I can find seems to give a hint as to what is using home. If I could get a solution that was not dependent on rebooting the machine, that would be ideal.

  • Is the file system in use because of your current login? If not, and you can't figure it out, just modify /etc/fstab so /home isn't mounted on boot and reboot. No, it's not going to let you figure out why it's behaving this way, but it will get you past the problem. – Andrew Henle Oct 9 '15 at 23:48
1

umount -f unfortunately just removes things from the VFS without actually umounting. It just doesn't appear anymore in /proc/mounts or anywhere, but it's still mounted. There is no obvious way to check what is still using this mount (or whether it is still there, as lsof and other things also fail at this, if you go by the pathname which no longer exists), short of checking the filesystem dirty flag and hoping this would be cleared once it's really umounted.

You can check some things manually, like devices or files in use by losetup, cat /proc/mdstat, dmsetup table, ... and check /proc/*/{maps,cwd,exe,fd/*} for references to the device in question (the mount path itself no longer exists). But that probably doesn't cover everything, either.

If you find such a reference (like a running process' open file descriptor) you'll find that you can access this file and modify it in any way you like thus proving that the filesystem is still mounted and fully operational, albeit invisible.

A bit of code I used to find such lazy umount presence, note it's hardcoded for a different device name.

isbusy() {
    grep -E '^/dev/mmcblk(0p3|1p1) ' /proc/mounts || \
    grep -E '^([^ ]+ ){3}b3:0[39] ' /proc/*/maps || \
    find /proc/[0-9]*/exe /proc/[0-9]*/cwd /proc/[0-9]*/fd -exec stat -tL {} + \
    | grep -E '^([^ ]+ ){6}b30[39]'
}

It's incomplete though.

1
  1. If at all possible, add more RAM to the system. Swap is not a solution, it's a temporary stop-gap fix to stop the system from falling over due to lack of memory. If a system swaps all the time then what it really needs is more RAM.

If the system can't have more RAM installed, replacing it with one that can would be worth considering.

  1. It's not as fast as a swap partition, because it has to go through the filesystem layer (you're unlikely to notice the difference), but you can create a large empty file on, say /home, and tell the system to use that as swap space. e.g.

    • dd if=/dev/zero of=/home/swapfile bs=1M count=16384

    • mkswap /home/swapfile

    • edit /etc/fstab and add /home/swapfile as more swap

    • swapon -a

  • I have used the filesystem based swapfile as a stop-gap for the time being. It is on a 5 disk RAID 0 partition so should be relatively quick if not as optimal as Linux could make it. I edited your response to add an additional command required to I guess 'format' the newly allocated space. – J Collins Nov 2 '15 at 16:43
0

I think you can try lvresize to reduce lv instead of using lvremove to remove lv. In this way you can avoid the problems lvremove command introduces. After using lvresize to reduce lv, you can use lvresize command to the swap to add volume from the centos VG.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.