I have a similar setup with Ubuntu 11.10. I made an Ubuntu live/installer CD and booted a laptop (a desktop would have worked as well) with the CD and the USB drive mounted. Then I just installed the OS onto the USB drive (being very careful not to install onto the laptop's hard drive). Now I can boot the laptop from the USB drive and have a running Ubuntu system that uses the USB drive for its main storage. And if I want to use the laptop's installed OS (happens to be Windows), I can just shut it down, remove the USB drive, and reboot.
EDIT : I found out today that this may not be as good an idea as I thought it was. I had initialized the USB thumb drive on one system (an Acer Aspire 5050 laptop), and until today I had never actually tried to use it on another system. Today I tried booting another laptop (a Dell Precision M2400) with the USB drive. It showed me a Debian (not Ubuntu) boot menu, and I never got a working desktop environment. It's still a (rather odd) way to have a dual-boot system without touching the hard drive, but it doesn't seem to be usable on systems other than the one where it was created. (Which makes sense; there are configurations that are specific to the system on which you install.)
EDIT 2: Or maybe that isn't the problem; it's not working on the original laptop either. I might try reinstalling the OS on the USB drive.
Note that you can also write the live/installer CD image directly to a USB drive, but that doesn't give you the same thing; it lets you use the USB drive as if it were a live CD, not a fully installed OS.