Manually cloning a Live USB Arch distro to a VMware virtual disk

Question

I installed Arch Linux on a pen drive with yumi, based on syslinux. I would like to test the USB distro in VMware. To speed up things I would like to clone the UFD as a VMware virtual disk (vmdk).

This can be simple. I attach to the virtual machine the physical UFD drive, the vmdk virtual disk and the Arch ISO. Bootstrapping from the latter, I can clone.

My problem is that the UFD is very large and I would like to copy/clone only the actual UFD used sectors.

I don't know if there are specific tools for cloning only used sectors. My idea is to manually format the vmdk disk and then copy the UFD files.

Given the following disk mapping:

Virtual vmdk disk -> sda
Physical UFD disk -> sdb

I issued these commands:

# Format vmdk disk as FAT32
(echo o; echo n; echo p; echo 1; echo ; echo ; echo a; echo t; echo c; echo w) | fdisk /dev/sda
mkdosfs -F 32 /dev/sda1

#Copy UFD MBR to vmdk MBR
dd if=/dev/sdb of=/dev/sda bs=446 count=1
partprobe /dev/sda

#Mount vmdk 
mkdir /vmdk
mount -t vfat /dev/sda1 /vmdk

#Mount UDF
mkdir /usb
mount -t vfat /dev/sdb1 /usb

#Copy files from UFD to vmdk
cp -a  /usb/* /vmdk

After them, fdisk -l gives:

Disk /dev/sda: 858 MB, 858783744 bytes, 1677312 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk label type: dos Disk identifier: 0xc3072e18

Device Boot Start End Blocks Id System /dev/sda1 * 2048 1677311 837632 c W95 FAT32 (LBA)

Disk /dev/sdb: 4048 MB, 4048551936 bytes, 7907328 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk label type: dos Disk identifier: 0xc3072e18

Device Boot Start End Blocks Id System /dev/sdb1 * 2976 7907327 3952176 c W95 FAT32 (LBA)

Disk /dev/mapper/arch_root-image: 1563 MB, 1563426816 bytes, 3053568 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes

Unfortunately, when I shutdown the VM, detach the UFD and the ISO, and restart the machine, VMware says:

This is not a bootable disk... 

Can you help me?

Answer 1

I think you can use one of the methods discussed in this U&L Q&A titled: How can I zero files out inside a VMware image file so that their space can be reclaimed? to zero out any unused space in the image prior, so that it can be reclaimed, prior so that the image size can be reduced.

Answer 2

Use tar. If you tarball your root dir it will only copy the files used and not the entire system. On your new system just replace your root directory and then re-install grub.

$ cd /
$ mkdir backups

$ tar -cvpf /backups/fullbackup.tar --directory=/ --exclude=proc --exclude=sys --exclude=dev/pts --exclude=backups . 

Then extract it in your new system (tar -xvpf /fullbackup.tar) and reinstall grub to the MBR.

$ grub-install recheck /dev/sdX (where x is whatever your MBR is)
$ grub-update

Answer 3

This is a partial, but I think very interesting, approach/solution.

Instead of copying Arch ISO to an UFD via YUMI and after copying the USB drive to a virtual disk, I can copy the Arch ISO straight to the virtual disk.

I bootstrap the VM with the Arch ISO. To copy the ISO to the virtual disk, assumed here sda, I have two possibilities. The first is formatting sda Fat32 (like the YUMI UFD):

#!/bin/sh -xv


# Partition and Format fat32 first VM disk
if (mountpoint -q /vdk); then umount /vdk; fi
echo "1,,c,*" | sfdisk /dev/sda
mkdosfs -F 32 /dev/sda1 

# Mount v-disk 
mkdir -p /vdk
mount -t vfat /dev/sda1 /vdk 

#Copy Arch ISO files in `arch' dir to v-disk
cp -a /run/archiso/bootmnt/arch /vdk

#Install Syslinux 
syslinux --directory /arch/boot/syslinux/ --install /dev/sda1

#Edit config files 
cd  /vdk/arch/boot/syslinux
sed -i "s|label=ARCH_.*|device=/dev/disk/by-uuid/$(blkid -o value -s UUID /dev/sda1)|" archiso_sys{32,64}.cfg

The second, perhaps more convenient, possibility is formatting Ext3:

#!/bin/sh -xv

# Partition and Format Ext3 first VM disk
if (mountpoint -q /vdk); then umount /vdk; fi
echo "1,,L,*" | sfdisk /dev/sda
mkfs.ext3 /dev/sda1

# Mount v-disk 
mkdir -p /vdk
mount /dev/sda1 /vdk

#Copy Arch ISO files in `arch' dir to v-disk
cp -a /run/archiso/bootmnt/arch /vdk

#Install Syslinux 
cd  /vdk/arch/boot/syslinux
extlinux --install . 
dd bs=440 conv=notrunc count=1 if=/usr/lib/syslinux/mbr.bin of=/dev/sda

#Edit config files 
sed -i "s|label=ARCH_.*|device=/dev/disk/by-uuid/$(blkid -o value -s UUID /dev/sda1)|" archiso_sys{32,64}.cfg

To avoid typing, one can add a secondary CD/DVD drive to the virtual machine and pack these scripts in an ISO.

Windows users might be pleased to know that the free VMWare player comes already with a mkisofs in its install directory. To make an ISO of the files into the dir archextra:

# mkisofs -joliet-long  -o archextra.iso archextra

Answer 4

The problem

The YUMI UFD boot loader (Syslinux) appears to have hard coded information so that it cannot be copied to the virtual disk MBR with:

dd if=/dev/sdb of=/dev/sda bs=446 count=1

The solution

Reinstall Syslinux straight to the virtual disk MBR with:

syslinux --directory /multiboot --install /dev/sda1

--directory is the directory where syslinux.cfg and the other Syslinux config files are located. By simple inspection of YUMI UFD one understands it is multiboot.

The Scripted Solution

#!/bin/sh -xv

# Partition and Format fat32 first VM disk
if (mountpoint -q /vdk); then umount /vdk; fi
echo "1,,c,*" | sfdisk /dev/sda
mkdosfs -F 32 /dev/sda1 

# Mount v-disk 
mkdir -p /vdk
mount -t vfat /dev/sda1 /vdk 

# Mount usb-disk 
mkdir -p /usb
if (mountpoint -q /usb); then umount /usb; fi
mount -t vfat /dev/sdb1 /usb

#Copy files from UFD to v-disk 
cp -a  /usb/* /vdk

#Install Syslinux relative to YUMI config file directory
syslinux --directory /multiboot --install /dev/sda1

Note YUMI passes to Arch (via Syslinux) the boot parameter archisolabel=MULTIBOOT. Currently, despite sda is not labelled MULTIBOOT, Arch will boot. Anyway you may want to make the script more robust appending:

mlabel -s -i /dev/sda1 ::MULTIBOOT

Further Investigation

Further investigation would show if it is YUMI or Syslinux MBR to be non-portable.