I have Ubuntu and I've installed VirtualBox in the desktop and installed CentOS but how would I go about connecting to it remotely via SSH as if I type my IP in? When I do this it just loads the Ubuntu SSH?
You have 2 methods for doing what you want.
Without having to get another physical IP address that is on your LAN, you can setup a port forward rule on your VirtualBox host, that will direct any traffic from say port 2222 to the port 22 of the IP address that has been designated for the VirtualBox guest.
The other method is to get your guest to have an IP address like any regular system would have on your LAN, this IP would then be accessible just like the IP of your VirtualBox host.
See here for further details.