Did you leave kernel debugging enabled? If you did that can yield a much larger RPM than when it is disabled.
I accepted the default options from menuconfig and ran
make rpm on the 184.108.40.206 kernel. The RPM it generated was 221M. After deselecting debugging, it is 21M. It was reduced by a factor of 10 which is close to the reduction you're expecting.
In 'menuconfig' you can disable kernel debugging by unchecking Kernel Hacking->Kernel debugging, or you can edit the config file directly and comment out CONFIG_DEBUG_KERNEL .
The RPM based distros have guides on building a kernel that might be helpful to you. Usually the recommendation is to not do it, but if you really need to do it, they have docs on how to do it using their method instead of the 'make rpm' method directly from the kernel tarball.