The normal way to manage installed applications under Linux is with a package manager. The choice of package managers is one of the main things that differentiate distributions. Ubuntu, like Debian (which it is based on), uses dpkg and APT; most of the time, you only need to interact with one of the interfaces to APT, such as
apt-get (command line), aptitude (command line or text mode) or Synaptic (GUI).
A package manager keeps track of which files belongs to which installed program. Like on Windows, programs can execute arbitrary code as part of their installation or uninstallation procedure, but are usually well-behaved and won't break other programs. Furthermore the (un)installation code is written by the package maintainer, not by the upstream author (for packages in the main distribution). Unlike Windows, there is a unified interface to installation, upgrade and uninstallation: the package manager. You don't need to search for the uninstaller (if there is one), you just click the “uninstall” icon in the graphical package manager, or run
apt-get remove PACKAGENAME.
If you need “exotic” software, you may need to install it manually, either by unpacking an archive or by compiling from source. Installers that come in the form of an executable program are rare in the Linux world. Running
make install tends to spread each program over several directories (
/usr/local/lib, etc.). To keep things sorted, I recommend using a “poor man's package manager”, such as stow. With stow, each package is installed in its own directory, and the
stow utility takes care of creating symbolic links so that the commands installed by the package are in the command search path and so on. See Keeping track of programs for more details.