Yes, you can do it but it is a bit of a tedious process.
First prepare the USB with the partitions you need to match your VM. After that mount the USB's partition/s you've created and
rsync (don't copy as I first posted since copying wont keep users and permissions) all files from your VM to the USB partition/s, avoid copying /dev /sys and /proc.
Once you've done that unmount the USB's partition/s and mount / in /mnt. After that mount with bind /dev, /sys and /proc:
mount --bind /dev /mnt/dev
mount --bind /proc /mnt/proc
mount --bind /sys /mnt/sys
Once you've done that, chroot to /mnt. In case /boot is a partition mount that partition now.
blkid and identify your USBs UUID root partition, edit /etc/fstab and modify the UUID of root for the one in your USB (and all mount points if you have various partitions).
At this point you must update your grub so that a new /boot/grub/grub.cfg is generated. I don't know if updating grub from the chrooted will create the right grub.cfg, but you can edit it manually and replace UUID with the right one, and once you start from your USB update-grub from there.
OK you're almost done now, install grub in the USB and you should be ready.
Make sure the root partition is set as bootable in your USB and try the USB.
I hope I haven't missed any step, but in case you have any problem let me know and I'll try to help.