WARNING: these operations is extremely risky, I give no warranty about it, it is recommended that you understand every single step and apply it at your own risk, if you don't you are probably gonna break your system and I am not liable for any damages arising from its use.
I have cloned all server disks on a local virtual machine on my workstation in order to try the commands without affecting my original server, if you are unsure you can do the same. After a extensive research and several experiments I finally found a solution.
Some systems uses systemd on initrd, after mounting the target root it switch the systemd environment the same way you would do with chroot, all services are shutdown and then restarted on the new environment.
We could take advantage of this feature to change the root again, doing so on a cloned environment on other disk will release the locks the system holds on the older environment, so you could safely umount these points - in fact it mostly will not even be mounted. But this cloned environment should guarantee the correct configuration to bring network up again and give ssh access. If it does not, consider yourself doomed.
It is important that you have a separated partition, once if you chroot to a directory inside other partition instead of a device itself the system can still hold the parent partition's device, for this use case we should consider it is intended it does not block any.
If you have unallocated free space, or you are able to shrink some partitions in order to have space for the cloned environment it is recommended. However if you does not have free storage space you can create a
tmpfs memory mount point if you have enough memory. A network partition is not recommended once the connection can get lost during the procedure.
Even though you does not have free storage space neither enough memory you can still create an disk image with a compressed btrfs partition inside a
tmpfs memory mount point that would allow you to write more data than you usually have of memory. In this case be careful when the partition turns read only, if it happens it probably starved.
It is important to note that doing chroot inside a memory partition can cause you to have this memory locked on the older environment such a way you are unable to free it without a
After having provisioned the required partitions for the cloned environment here how it is going to work:
$ sudo mount /dev/clone_environment_device /mnt
# mount other separate partitions if required
$ sudo mkdir -p /mnt/mnt /mnt/dev /mnt/proc /mnt/sys /mnt/tmp /mnt/var/tmp /mnt/run
$ sudo chmod 1777 /mnt/tmp /mnt/var/tmp
$ sudo ln -s ../run /mnt/var/run
$ sudo rsync -aAHXSv \
--exclude 'mnt' \
--exclude 'dev' \
--exclude 'run' \
--exclude 'sys' \
--exclude 'proc' \
--exclude 'tmp' \
--exclude 'var/tmp' \
It is important to adjust the devices path and uuid (
blkid) to match the new ones.
$ sudo vi /mnt/etc/fstab
$ sudo vi /mnt/etc/default/grub
Then, to perform the chroot:
$ sudo mkdir /sysroot
$ sudo mount --rbind /mnt /sysroot
$ sudo touch /etc/initrd-release
$ sudo systemctl --no-block isolate initrd-switch-root
# it will stop all other services (isolate) and call systemctl switch-root /sysroot
Note that it is not required to bind
dev as you normally do when performing a chroot, the systemd will do it for you. You may lost connection, if you wait some time and it still be unable to connect you have my condolences, you have been wasted.
Though if you were lucky and be able to connect do not forget to install the bootloader for the new environment in order to make it up again in case of a shutdown:
$ sudo grub2-install /dev/intended_bootable_disk
$ sudo grub2-mkconfig -o /etc/grub2.cfg
Of course, do it only if you are not on the memory mount point, if you are, you now must erase the other partitions and do it all again with the real storage.
Remember, do it at your own risk.