0

I have a binary that I can compile with either clang or gcc, and the directory for the application is something like this:

/app root dir
  /bin
    /<application>
  /lib
    /<application specific libs>
  /syslib
    /<system libs, e.g. boost libraries>

The binary is compiled with (amongst others) the following options for both compilers:

-Wl,-rpath,../syslib -ljansson -lsqlite3 -lboost_date_time -lboost_filesystem -lboost_log -lboost_program_options -lboost_thread -lboost_system

So during execution, the application should be able to find the provided system libraries. However, when I use the binary compiled with g++, it isn't able to find a specific library. The libraries available in the syslib directory are:

 - libboost_date_time.so -> libboost_date_time.so.1.65.1
 - libboost_date_time.so.1.65.1 libboost_filesystem.so ->
 - libboost_filesystem.so.1.65.1 libboost_filesystem.so.1.65.1
 - libboost_log.so -> libboost_log.so.1.65.1 libboost_log.so.1.65.1
 - libboost_program_options.so -> libboost_program_options.so.1.65.1
 - libboost_program_options.so.1.65.1 libboost_system.so ->
 - libboost_system.so.1.65.1 libboost_system.so.1.65.1
 - libboost_thread.so -> libboost_thread.so.1.65.1
 - libboost_thread.so.1.65.1 libjansson.so -> libjansson.so.4.10.0
 - libjansson.so.4 -> libjansson.so.4.10.0 libjansson.so.4.10.0
 - libsqlite3.so -> libsqlite3.so.0.8.6 libsqlite3.so.0 ->
 - libsqlite3.so.0.8.6 libsqlite3.so.0.8.6 libstdc++.so ->
 - libstdc++.so.6.0.24 libstdc++.so.6 -> libstdc++.so.6.0.24
 - libstdc++.so.6.0.24

When I execute the g++ compiled binary, it can't find the libboost_thread library while with the clang++ compiled binary, it works fine. Errror output from binary:

./DGM: error while loading shared libraries: libboost_thread.so.1.65.1: cannot open shared object file: No such file or directory

ldd output for g++ compiled binary:

linux-vdso.so.1 =>  (0x00007fff7c90c000)
libjansson.so.4 => ../syslib/libjansson.so.4 (0x00007fea61f7b000)
libboost_date_time.so.1.65.1 => ../syslib/libboost_date_time.so.1.65.1 (0x00007fea61d69000)
libboost_filesystem.so.1.65.1 => ../syslib/libboost_filesystem.so.1.65.1 (0x00007fea61b4d000)
libboost_log.so.1.65.1 => ../syslib/libboost_log.so.1.65.1 (0x00007fea6186c000)
libboost_program_options.so.1.65.1 => ../syslib/libboost_program_options.so.1.65.1 (0x00007fea615e9000)
libboost_system.so.1.65.1 => ../syslib/libboost_system.so.1.65.1 (0x00007fea613e3000)
libstdc++.so.6 => ../syslib/libstdc++.so.6 (0x00007fea6105d000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fea60d51000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fea60b39000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fea6091b000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fea60556000)
libnetsnmpagent.so.30 => /usr/lib/x86_64-linux-gnu/libnetsnmpagent.so.30 (0x00007fea602ec000)
libnetsnmp.so.30 => /usr/lib/x86_64-linux-gnu/libnetsnmp.so.30 (0x00007fea60012000)
libsqlite3.so.0 => ../syslib/libsqlite3.so.0 (0x00007fea5fd09000)
--> libboost_thread.so.1.65.1 => not found <--
librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007fea5fb01000)
/lib64/ld-linux-x86-64.so.2 (0x00007fea639f3000)
libwrap.so.0 => /lib/x86_64-linux-gnu/libwrap.so.0 (0x00007fea5f8f6000)
libperl.so.5.18 => /usr/lib/libperl.so.5.18 (0x00007fea5f56d000)
libcrypto.so.1.0.0 => /lib/x86_64-linux-gnu/libcrypto.so.1.0.0 (0x00007fea5f193000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fea5ef8e000)
libnsl.so.1 => /lib/x86_64-linux-gnu/libnsl.so.1 (0x00007fea5ed74000)
libcrypt.so.1 => /lib/x86_64-linux-gnu/libcrypt.so.1 (0x00007fea5eb3b000)

ldd output for clang++ compiled binary:

linux-vdso.so.1 =>  (0x00007fff0cdc5000)
libjansson.so.4 => ../syslib/libjansson.so.4 (0x00007fca078f9000)
libsqlite3.so.0 => ../syslib/libsqlite3.so.0 (0x00007fca075f1000)
libboost_date_time.so.1.65.1 => ../syslib/libboost_date_time.so.1.65.1 (0x00007fca073de000)
libboost_filesystem.so.1.65.1 => ../syslib/libboost_filesystem.so.1.65.1 (0x00007fca071c3000)
libboost_log.so.1.65.1 => ../syslib/libboost_log.so.1.65.1 (0x00007fca06ee2000)
libboost_program_options.so.1.65.1 => ../syslib/libboost_program_options.so.1.65.1 (0x00007fca06c5e000)
--> libboost_thread.so.1.65.1 => ../syslib/libboost_thread.so.1.65.1 (0x00007fca06a38000) <--
libboost_system.so.1.65.1 => ../syslib/libboost_system.so.1.65.1 (0x00007fca06832000)
libstdc++.so.6 => ../syslib/libstdc++.so.6 (0x00007fca064ac000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fca061a0000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fca05f89000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fca05d6a000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fca059a5000)
libnetsnmpagent.so.30 => /usr/lib/x86_64-linux-gnu/libnetsnmpagent.so.30 (0x00007fca0573b000)
libnetsnmphelpers.so.30 => /usr/lib/x86_64-linux-gnu/libnetsnmphelpers.so.30 (0x00007fca05539000)
libnetsnmp.so.30 => /usr/lib/x86_64-linux-gnu/libnetsnmp.so.30 (0x00007fca0525f000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fca0505a000)
librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007fca04e52000)
/lib64/ld-linux-x86-64.so.2 (0x00007fca08a94000)
libwrap.so.0 => /lib/x86_64-linux-gnu/libwrap.so.0 (0x00007fca04c47000)
libperl.so.5.18 => /usr/lib/libperl.so.5.18 (0x00007fca048be000)
libcrypto.so.1.0.0 => /lib/x86_64-linux-gnu/libcrypto.so.1.0.0 (0x00007fca044e4000)
libnsl.so.1 => /lib/x86_64-linux-gnu/libnsl.so.1 (0x00007fca042c9000)
libcrypt.so.1 => /lib/x86_64-linux-gnu/libcrypt.so.1 (0x00007fca04090000)

I find this exceptionally weird as the g++ compiled binary can find other boost libraries in the same directory.

As requested, the strace output of the g++ compiled binary for libboost_thread.so compared with libboost_program_options.so:

libboost_program_options.so:

getcwd("/opt/dap/apps/DGM/bin", 128)    = 22
mmap(NULL, 3016424, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f0a5558d000
mprotect(0x7f0a55666000, 2097152, PROT_NONE) = 0
mmap(0x7f0a55866000, 32768, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0xd9000) = 0x7f0a55866000
close(3)                                = 0
open("../lib/tls/x86_64/libboost_program_options.so.1.65.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("../lib/tls/libboost_program_options.so.1.65.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("../lib/x86_64/libboost_program_options.so.1.65.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("../lib/libboost_program_options.so.1.65.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("../syslib/tls/x86_64/libboost_program_options.so.1.65.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("../syslib/tls/libboost_program_options.so.1.65.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("../syslib/x86_64/libboost_program_options.so.1.65.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("../syslib/libboost_program_options.so.1.65.1", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\300\233\2\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=667552, ...}) = 0

libboost_thread.so:

getcwd("/opt/dap/apps/DGM/bin", 128)    = 22
mmap(NULL, 3175320, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f0a53a34000
mprotect(0x7f0a53b36000, 2097152, PROT_NONE) = 0
mmap(0x7f0a53d36000, 20480, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x102000) = 0x7f0a53d36000
mmap(0x7f0a53d3b000, 920, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f0a53d3b000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/tls/x86_64/libboost_thread.so.1.65.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
stat("/lib/x86_64-linux-gnu/tls/x86_64", 0x7fffc0976a60) = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/tls/libboost_thread.so.1.65.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
stat("/lib/x86_64-linux-gnu/tls", 0x7fffc0976a60) = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/x86_64/libboost_thread.so.1.65.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
stat("/lib/x86_64-linux-gnu/x86_64", 0x7fffc0976a60) = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libboost_thread.so.1.65.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
stat("/lib/x86_64-linux-gnu", {st_mode=S_IFDIR|0755, st_size=12288, ...}) = 0
open("/usr/lib/x86_64-linux-gnu/tls/x86_64/libboost_thread.so.1.65.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
stat("/usr/lib/x86_64-linux-gnu/tls/x86_64", 0x7fffc0976a60) = -1 ENOENT (No such file or directory)
open("/usr/lib/x86_64-linux-gnu/tls/libboost_thread.so.1.65.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
stat("/usr/lib/x86_64-linux-gnu/tls", 0x7fffc0976a60) = -1 ENOENT (No such file or directory)
open("/usr/lib/x86_64-linux-gnu/x86_64/libboost_thread.so.1.65.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
stat("/usr/lib/x86_64-linux-gnu/x86_64", 0x7fffc0976a60) = -1 ENOENT (No such file or directory)
open("/usr/lib/x86_64-linux-gnu/libboost_thread.so.1.65.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
stat("/usr/lib/x86_64-linux-gnu", {st_mode=S_IFDIR|0755, st_size=12288, ...}) = 0
open("/lib/tls/x86_64/libboost_thread.so.1.65.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
stat("/lib/tls/x86_64", 0x7fffc0976a60) = -1 ENOENT (No such file or directory)
open("/lib/tls/libboost_thread.so.1.65.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
stat("/lib/tls", 0x7fffc0976a60)        = -1 ENOENT (No such file or directory)
open("/lib/x86_64/libboost_thread.so.1.65.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
stat("/lib/x86_64", 0x7fffc0976a60)     = -1 ENOENT (No such file or directory)
open("/lib/libboost_thread.so.1.65.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
stat("/lib", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
open("/usr/lib/tls/x86_64/libboost_thread.so.1.65.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
stat("/usr/lib/tls/x86_64", 0x7fffc0976a60) = -1 ENOENT (No such file or directory)
open("/usr/lib/tls/libboost_thread.so.1.65.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
stat("/usr/lib/tls", 0x7fffc0976a60)    = -1 ENOENT (No such file or directory)
open("/usr/lib/x86_64/libboost_thread.so.1.65.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
stat("/usr/lib/x86_64", 0x7fffc0976a60) = -1 ENOENT (No such file or directory)
open("/usr/lib/libboost_thread.so.1.65.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
stat("/usr/lib", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
writev(2, [{"./DGM", 5}, {": ", 2}, {"error while loading shared libra"..., 36}, {": ", 2}, {"libboost_thread.so.1.65.1", 25}, {": ", 2}, {"cannot open shared object file", 30}, {": ", 2}, {"No such file or directory", 25}, {"\n", 1}], 10./DGM: error while loading shared libraries: libboost_thread.so.1.65.1: cannot open shared object file: No such file or directory
) = 130
exit_group(127)                         = ?
+++ exited with 127 +++
  • The problem appears to be how your build is generating library references. Note the reference... libboost_thread.so.1.65.1 => ../syslib/libboost_thread.so.1.65.1 which contains a relative path, rather than an absolute path, which means this executable must run out of a specific directory + will fail when run from any where else. This reminds me of problems I've seen in the past building with libtool, where the final libtool step of rewriting all library links was missed. Look at your entire build process for how to fix this. – David Favor Oct 25 '17 at 10:43
  • It would be interesting to run strace and see what files the loader tries to access during the failed attempt to load that one library. – Mark Plotnick Oct 25 '17 at 10:54
  • It's probably better to use the option '-Wl,-rpath,$ORIGIN/../syslib' (with the single quotes) instead of -Wl,-rpath,../syslib, although that probably won't help your specific problem. – Mark Plotnick Oct 25 '17 at 10:58
  • @MarkPlotnick: where does the $ORIGIN variable come from? I don't recognize it. – ThaMe90 Oct 26 '17 at 9:32
  • @MarkPlotnick: I added the strace output for a library it can find and the library it can't find. – ThaMe90 Oct 26 '17 at 10:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.