Configure Python to include another directory when looking for packages

Question

I'm on a SUSE machine where the default Python site-packages location is /usr/lib64/python2.6/site-packages. Some packages automatically install themselves in /usr/lib/python2.6/site-packages instead. How do I configure Python so that it also looks in /usr/lib64/python2.6/site-packages?

Answer 1

(Please correct errors and omissions as necessary. Thanks.)

First, a question and a comment. I don't use SUSE, so take this with a pinch of salt. Are the packages that install in /usr/lib/python2.6/site-packages official packages? If so, SUSE is broken, so that is not likely. If they are not official packages, you could either ask the packagers to use the standard paths, or, alternatively, you could submit a wishlist bug to SUSE asking them to support this additional path. This will save you and other people additional headaches.

For the moment, you have the following possibilities, in order of decreasing scope:

1. Change the module search path for all users (method 1)

   Change the module search path in the Python installation.

   The default module search path is hardwired into the binary. Add-on paths can be configured at runtime, for example in the site.py file. For example, Debian uses /usr/lib/python2.6/site.py (for the default python 2.6 installation) to do its site-specific configuration. At the top of the file is written

   Append module search paths for third-party packages to `sys.path`.
   
   ****************************************************************
   * This module is automatically imported during initialization. *
   ****************************************************************
   [...]
   
   This will append site-specific paths to the module search path.
   

   The Debian patch debian/patches/site-locations.diff says

   For Debian and derivatives, this sys.path is augmented with directories for packages distributed within the distribution. Local addons go into /usr/local/lib/python/dist-packages, Debian addons install into /usr/{lib,share}/python/dist-packages. /usr/lib/python/site-packages is not used.

   The patch in question is

                   sitedirs.append(os.path.join(prefix,
   "Lib", "site-packages"))
                elif os.sep == '/':
       +            sitedirs.append(os.path.join(prefix,
   "local/lib",
       +                                        "python" + sys.version[:3],
       +                                        "dist-packages"))
                    sitedirs.append(os.path.join(prefix,
   "lib",
                                                "python" + sys.version[:3],
       -                                        "site-packages"))
       -            sitedirs.append(os.path.join(prefix,
   "lib", "site-python"))
       +                                        "dist-packages"))
       +            sitedirs.append(os.path.join(prefix,
   "lib", "dist-python"))
                else:
                    sitedirs.append(prefix)
                    sitedirs.append(os.path.join(prefix,
   "lib", "site-packages"))
   

   So you could modify the site.path in your system package to produce a modified module search path. You probably don't want to this, though. For one thing, you will have to merge this in on every update of your distribution's python package.

2. Change the module search path for all users (method 2)

   Add a file of the form something.pth to a directory that is already in the search path, which contains a path, either relative or absolute. Eg.

   /usr/local/lib/python2.6/dist-packages$ cat foo.pth 
   /home/faheem/dummypythonpath
   

   In another terminal do

   >>> import sys
   >>> sys.path
       [...,'/home/faheem/dummypythonpath',...,]
   

3. Change the module search path for all users (method 3)

   The environment variable PYTHONPATH is normally used to append to the system path at user level. You can put it in a file which will be sourced by all users. Eg. in Debian we have /etc/bash.bashrc, which says at the top

   # System-wide .bashrc file for interactive bash(1) shells.
   
   # To enable the settings / commands in this file for login shells as well,
   # this file has to be sourced in /etc/profile.
   

   So you could add or PYTHONPATH there. You probably want it to be sourced for both login and interactive shells, so you'll want to check on that. Unfortunately, distributions are often flaky about enabling this.

   The paths in PYTHONPATH are added to the default list of search paths in the system (which can be obtained for example by sys.path - see below). Allowing for the possibility that PYTHONPATH is set already, just add desired additional directories to it, eg.

   export PYTHONPATH=$PYTHONPATH:"/home/username/lib/python2.6/dist-packages"
   

   If you source the PYTHONPATH variable, and then check sys.path again, you will
   see the paths have been added. Note that the position in which the paths in
   PYTHONPATH are added to the pre-existing paths does not seem to be prescribed by
   the implementation.

4. Change the module search path per user.

   The usual way is to change PYTHONPATH in the user's bashrc, namely ~/.bashrc. Again, check that it is sourced for both login and interactive shells.

5. Change the module search path on a per script basis. This is done by appending to sys.path, namely

   import sys
   sys.path.append(some_additional_path)
   

   This will only work for the script that is importing this. This is normally used, as far as I know, for casual use, when importing modules in nonstandard locations, like from somewhere in a home directory.

See also Greg Ward on Modifying Python's Search Path. This has a good discussion of the available alternatives.

Answer 2

Use sys.path:

import sys
sys.path.append("/home/me/mypy")

You can also check the site module documentation, which explains how the site-specific paths are computed.