Unchecking filesystems and drivers isn't going to reduce the size of the kernel at all, because they are compiled as modules and only the modules that correspond to hardware that you have are loaded.
There are a few features of the kernel that can't be compiled as modules and that you might not be using. Start with Ubuntu's .config, then look through the ones that are compiled in the kernel (y, not m). If you don't understand what a feature is for, leave it alone.
Most of the kernel's optional features are optional because you might not want them on an embedded system. Embedded systems have two characteristics: they're small, so not wasting memory on unused code is important, and they have a dedicated purpose, so there are many features that you know you aren't going to need. A PC is a general-purpose device, where you tend to connect lots of third-party hardware and run lots of third-party software. You can't really tell in advance that you're never going to need this or that feature. Mostly, what you'll be able to do without is support for CPU types other than yours and workarounds for bugs in chipsets that you don't have (what few aren't compiled as modules). If you compile a 64-bit kernel, there won't be a lot of those, not nearly as many as a 32-bit x86 kernel where there's quite a bit of historical baggage.
In any case, you are not going to gain anything significant. With 8GB of memory, the memory used by the kernel is negligible.
If you really want to play around with kernels and other stuff, I suggest getting a hobbyist or utility embedded board (BeagleBoard, Gumstix, SheevaPlug, …).
rxvtand so on. – dmckee Nov 6 '11 at 19:44