Keeping binaries in the directories with matching libraries is quite common. The use case is version control. Say when you have several versions of python (2.7, 2.7, 3...) installed and need them all but want to define a standard.
Typically this uses the
alternatives suite and softlinks to the binary from e.g.
/usr/bin (with definitions in
One major advantage might also be using the left to right order in
$PATH. E.g. when you override standard commands by local versions or scripts that perform prechecks (say a script that puts a warning before
shredding files), one would have them in a directory that is listed at the beginning of
$PATH, so the local (user's) version is run and the standard one only via its full path.
More on your question: Moving the binaries is not a good idea: they may rely on other files in their directory. Soft link them at most. Or just use the
alternatives set of tools - which also helps in keeping more overview.