3

I am using Docker with image of Ubuntu 16.04. I am using this docker to run:

  • Import C++ classes in python with the help of boost.python
  • Import matlab's python package in python with the help Matlab Runtime (MCR).

Before I have installed MCR in the docker, I have installed boost using:

$ sudo apt-get install -y libboost-all-dev

and I was able to wrap C++ classes and call them in python. Then I have installed MCR and one of the requirement is to set LD_LIBRARY_PATH to Matlab's library.

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/MATLAB/MATLAB_Runtime/v94/runtime/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v94/bin/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v94/sys/os/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v94/extern/bin/glnxa64

It worked, I was able to run matlab's python package in python but testing the C++ classes wrappers failed due to change of LD_LIBRARY_PATH. For example, I get the following error when I try to import C++ class in python:

 /usr/lib/x86_64-linux-gnu/libpython3.4m.so.1.0: undefined symbol: XML_SetHashSalt

If I unset LD_LIBRARY_PATH C++ import works but MCR fails. So the question what LD_LIBRARY_PATH should be in order to run both C++ classes and Matlab's python package successfully?

I have tried to add path to boost libraries to LD_LIBRARY_PATH but didn't work, so the env value would be:

/usr/lib/x86_64-linux-gnu:/usr/local/MATLAB/MATLAB_Runtime/v94/runtime/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v94/bin/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v94/sys/os/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v94/extern/bin/glnxa64

EDIT

the dependencies of C++ library file before setting the LD_LIBRARY_PATH:

    linux-vdso.so.1 (0x00007ffcee0dc000)
    libpython3.4m.so.1.0 => /usr/lib/x86_64-linux-gnu/libpython3.4m.so.1.0 (0x00007f9d69e59000)
    libboost_python-py34.so.1.55.0 => /usr/lib/x86_64-linux-gnu/libboost_python-py34.so.1.55.0 (0x00007f9d69c09000)
    libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f9d698fe000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f9d695fd000)
    libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f9d693e7000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f9d6903c000)
    librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f9d68e34000)
    libexpat.so.1 => /lib/x86_64-linux-gnu/libexpat.so.1 (0x00007f9d68c0b000)
    libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f9d689f0000)
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f9d687d3000)
    libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f9d685cf000)
    libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007f9d683cc000)
    /lib64/ld-linux-x86-64.so.2 (0x00007f9d6a6c3000)

After setting the LD_LIBRARY_PATH:

    linux-vdso.so.1 (0x00007ffc42e9b000)
    libpython3.4m.so.1.0 => /usr/lib/x86_64-linux-gnu/libpython3.4m.so.1.0 (0x00007fad9635b000)
    libboost_python-py34.so.1.55.0 => /usr/lib/x86_64-linux-gnu/libboost_python-py34.so.1.55.0 (0x00007fad9610b000)
    libstdc++.so.6 => /usr/local/MATLAB/MATLAB_Runtime/v94/sys/os/glnxa64/libstdc++.so.6 (0x00007fad95d8a000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fad95a89000)
    libgcc_s.so.1 => /usr/local/MATLAB/MATLAB_Runtime/v94/sys/os/glnxa64/libgcc_s.so.1 (0x00007fad95873000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fad954c8000)
    librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007fad952c0000)
    libexpat.so.1 => /usr/local/MATLAB/MATLAB_Runtime/v94/bin/glnxa64/libexpat.so.1 (0x00007fad95095000)
    libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007fad94e7a000)
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fad94c5d000)
    libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fad94a59000)
    libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007fad94856000)
    /lib64/ld-linux-x86-64.so.2 (0x00007fad96bc5000)
  • LD_LIBRARY_PATH should be avoided in favor of binaries that set the RUNPATH internally in the ELF header. – schily Aug 15 '18 at 13:53
  • 2
    I'm not sure why this is getting downvoted - aside from the fact that it's arguably up to Mathworks to distribute their application in a way that doesn't require such hackery. You may find some of the discussion here relevant: MATLAB bindings, when installed, do not work – steeldriver Aug 15 '18 at 19:19
  • 1
    @steeldriver thanks for your support. I really hate it when people downvote without pointing out what is the problem. Anyway, thanks for the link I will have a look at it – Coderji Aug 16 '18 at 7:37
  • 1
    I'm sorry if this is a weird question, but do you need both of these things to work at the same time (i.e. you run one command and both dependencies are needed) or are you able to run these things separately (i.e. you could run var=whatever command to launch a command with a certain var set without setting it for the whole session) ... If the latter, then you could make a .sh script that runs the command that needs the variable with var=whatever in front of it and not have it set the rest of the time. – 3D1T0R Aug 31 '18 at 22:24
  • 1
    unset LD_LIBRARY_PATH – mikeserv Sep 1 '18 at 0:44
2
+100

You have a couple of options:

  1. Prefix the path to your shared libraries in the export statement above (more than likely $LD_LIBRARY_PATH is not set on your system and so the only paths that are getting set are the ones explicitly set in your export statement.)

    export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib/:/usr/lib/:/usr/lib64/:/usr/local/MATLAB/MATLAB_Runtime/v94/runtime/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v94/bin/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v94/sys/os/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v94/extern/bin/glnxa64
    

    This export statement will not be a permanent part of your shell sessions unless you add it to .bashrc fyi.

  2. Add the MATLAB library path to /etc/ld.so.conf.d/ and run ldconfig to identify the new paths for the linker on your system.

    (BTW I am guessing at the MATLAB library name. You might need to make the name of the file something other than MATLAB like all lower case or something else. Basically we need the name of the library as it is referenced during the linking process. You could inspect a make file to see how this is referenced.)

    # Add the file '/etc/ld.so.conf.d/MATLAB' with the following lines 
    /usr/local/MATLAB/MATLAB_Runtime/v94/runtime/glnxa64
    /usr/local/MATLAB/MATLAB_Runtime/v94/bin/glnxa64
    /usr/local/MATLAB/MATLAB_Runtime/v94/sys/os/glnxa64
    /usr/local/MATLAB/MATLAB_Runtime/v94/extern/bin/glnxa64
    

    Then run ldconfig.

    sudo ldconfig
    

    You may need to unset the $LD_LIBRARY_PATH variable to make the linker use the normal system search path and pickup the new search path for MATLAB.

    unset $LD_LIBRARY_PATH
    
  • Thanks for your answer.. I will test what have you suggested and let you know. – Coderji Sep 4 '18 at 16:12
  • 1
    Thanks for your answer.. Your answer was the closest answer I didn't get it to work yet but I think I am getting there. Thanks again. – Coderji Sep 6 '18 at 15:10
  • Anything else needed to get this answer accepted? – datUser Sep 8 '18 at 15:18
0

I had similar issues on my workstation. To get around that, I had created a folder under this path /usr/lib/GLops, then I extracted the GL libraries from the Mesa rpms and put them in that directory. Finally, I added the following lines to ~.tcshrc

setenv LD_LIBRARY_PATH /usr/lib64/GLops/libGL.so
setenv LIBGL_ALWAYS_SOFTWARE 1
setenv LIBGL_ALWAYS_INDIRECT 1
setenv LIBGL_DIR3_DISABLE 1
setenv MESA_GL_VERSION_OVERRIDE 2.1

The path /usr/lib64/GLops/ contain the following files:

libGL.so  libGL.so.1  libGL.so.1.2.0  libxcb.so  libxcb.so.1  libxcb.so.1.1.0

The libs I have in /usr/lib64/GLops:/usr/lib/GLops

0

The only way it worked for me is by defining LD_PRELOAD and point it to system library. So my current environment would be:

LD_LIBRARY_PATH=/usr/local/MATLAB/MATLAB_Runtime/v94/runtime/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v94/bin/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v94/sys/os/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v94/extern/bin/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v94/sys/opengl/lib/glnxa64
LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libexpat.so

So in docker, I have created a .matlab file that contains the mentioned env variables. Then, in dockerfile I added the following:

services:
  django: &django
    build:
      context: .
      dockerfile: ./compose/local/django/Dockerfile
    depends_on:
      - postgres
    volumes:
      - .:/app
      - shared_upload_tmp:/app_temp
      - media:/app/computation_server/media
    env_file:
      - ./.envs/.local/.django
      - ./.envs/.local/.postgres
      - ./.envs/.production/.matlab
    ports:
      - "8800:8000"
    command: /start

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.