Although everyone uses compilation in the colloquial sense of turning source code into an executable it's technically a single step in a rather long pipeline:
- The input file is run through the preprocessor resulting in a single translation unit.
- The output of the preprocessor is compiled to assembly.
- The assembler takes that as input an outputs an object file.
- The linker stitches together the object files to produce an executable.
[ To be pedantic there's no requirement the steps be separate and modern compilers typically combine them for efficiency. ]
Our concern is the linking step, which combines your code with standard system libraries. The linker copies objects from static libraries directly into the executable. For shared libraries, however, it only provides a reference to the library.
Shared libraries have a lot of advantages. You can update them without recompiling programs and they use less memory because programs can share common code. They also have the obvious drawback that the code isn't in the executable.
The solution to that is the dynamic loader, which is responsible for resolving all shared library references at runtime. The loader is run automatically; instructions for doing so are one thing the linker includes in the executable. Of course this presupposes that the loader can find the libraries.
System libraries are in standard directories, which is straightforward. When that isn't the case the loader will search LD_LIBRARY_PATH. Why doesn't the linker just put the path in the executable? Because then you'd be unable to move or change the library.
In practice you couldn't really move the executable either, since the library is outside the system search path. If it only ran when the library was located in ~luke/lib then you can't give it to joe unless he can read your files. Sucks for joe if you move onto a new job.
Just FYI it would suck in a myriad of other ways as well. It'd make debugging an everlasting nightmare if you could only specify the library location at compile time, amongst other things.