0

Python pip and many Python modules are provided through the CentOS Linux repos. On CentOS 7.x Linux, I can install pip and pymongo with the standard distribution package manager (yum install python-pip python-pymongo).

However, the version of pymongo which is available from the repo is already too old. The version of pip is usually behind, too, so it immediately recommends running pip install --upgrade pip.

Is there a consensus on what one should do here? Using pip to run updates will be overwriting files that came from the RPMs, so you're likely to get complaints about that from the package manager.

Is it better not to install pip and the modules from RPM? Perhaps just installing Python from RPM and then installing all modules from pip (so that none of the Python modules are from RPM and are wholly managed by pip)?

  • IMO it is always best to stay within the packaging system. Sometimes this will mean building your own package of an updated (or unpackaged) program or library or module. – cas Apr 23 '16 at 2:31
2

Take a look at this question. It talks about having pip install in a different directory. This would allow you to maintain the rpm installs and have the latest version. I would probably uninstall the rpm version if you don't plan to use it.

https://stackoverflow.com/questions/2915471/install-a-python-package-into-a-different-directory-using-pip

| improve this answer | |
2

If you want to be able to use python just for yourself without messing up your system's python, take a look at a product called Anaconda from Continuum. http://Continuum.io/downloads this install of python etc... will install in your home directory and change paths so you're not installing to your systems python and possibly messing up your system. You can also make virtual environments and Conda environments of these options allow you to try something out and if you don't like it just blow away the environment.

| improve this answer | |
2

Is it better not to install pip and the modules from RPM? Perhaps just installing Python from RPM and then installing all modules from pip (so that none of the Python modules are from RPM and are wholly managed by pip)?

It is 'ok' to install pip and other Python modules from your OS's package manager. You may no longer have a choice about it since a lot of system components depend on Python and even in a minimal install version of the OS, you will find Python + Python modules installed as default.

So whenever you feel that you need to upgrade the 'system-wide' version of pip or any of the Python modules, but don't want to interfere with the files managed by the package manager, you should do the following:

The following instructions are for a RHEL derivative OS and the Python version available at the time was 2.7.x. You will need to adjust the PYTHONPATH to match your installed version of Python.

  • Ensure that the package manager version of python-pip is installed. e.g. yum -y install python-pip.

  • Create a profile.d script to set up the PYTHONPATH environment variable:

    # Ensure PYTHONPATH is configured to use /usr/local/lib*/python2.7/site-packages
    read -r -d '' pythonpath_profile_script <<'EOF'
    pythonpathmunge () {
        case ":${PYTHONPATH}:" in
            *:"$1":*)
                ;;
            *)
                if [ "$2" = "after" ] ; then
                    PYTHONPATH=$PYTHONPATH:$1
                else
                    PYTHONPATH=$1:$PYTHONPATH
                fi
        esac
    }
    
    pythonpathmunge /usr/local/lib/python2.7/site-packages
    pythonpathmunge /usr/local/lib64/python2.7/site-packages
    export PYTHONPATH
    EOF
    
    echo "Creating the /etc/profile.d/pythonpath.sh profile script:"
    echo "${pythonpath_profile_script}" > /etc/profile.d/pythonpath.sh
    chown root.root /etc/profile.d/pythonpath.sh
    chmod -v 0644 /etc/profile.d/pythonpath.sh
    source /etc/profile.d/pythonpath.sh
    hash -r
    
  • You can then choose to install the latest version of pip or any other python-module via:

    pip install --upgrade <<PYTHON MODULE NAME>> --ignore-installed --install-option="--prefix=/usr/local" --log /var/log/<<PYTHON MODULE NAME>>-install-$(date "+%Y%m%d%H%M%S").log
    

    e.g.

    pip install --upgrade pip --ignore-installed --install-option="--prefix=/usr/local" --log /var/log/pip-install-$(date "+%Y%m%d%H%M%S").log
    hash -r
    # Notice the use of `hash -r` to refresh bash's hash table of recently
    # executable programs. Without doing this you may find that the 'pip'
    # command being used is not the version you just installed.
    

This will install the requested Python module under /usr/local where it won't interfere with the files managed by the system package manager. The --ignore-installed option ensures pip doesn't touch the existing version of the module. Also, due to the use of the PYTHONPATH environment variable, the /usr/local/lib*/python2.7/site-packages paths will be used before any system installed packages. i.e.

[root@localhost ~]# python
Python 2.7.5 (default, Sep 15 2016, 22:37:39)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> print '\n'.join(sys.path)

/usr/local/lib64/python2.7/site-packages        <---
/usr/local/lib/python2.7/site-packages          <---
/root
/usr/lib64/python27.zip
/usr/lib64/python2.7
/usr/lib64/python2.7/plat-linux2
/usr/lib64/python2.7/lib-tk
/usr/lib64/python2.7/lib-old
/usr/lib64/python2.7/lib-dynload
/usr/lib64/python2.7/site-packages
/usr/lib/python2.7/site-packages
>>>

Lastly, at least on CentOS/RHEL, the /usr/local/bin path is before any of the other system binary paths in the PATH environment variable and hence this also ensures that any newer binaries installed in /usr/local/bin take precendence above system installed ones. i.e.

[root@localhost ~]$ echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin
| improve this answer | |
1

CentOS 7.x is derived from Red Hat Enterprise 7.x, which is designed to be stable. That inevitably means that the programs which are available from the repositories have been tested for a while. You can see that from the description on their website:

The CentOS Project is a community-driven free software effort focused on delivering a robust open source ecosystem. For users, we offer a consistent manageable platform that suits a wide variety of deployments. For open source communities, we offer a solid, predictable base to build upon, along with extensive resources to build, test, release, and maintain their code.

If you want "bleeding-edge" code, you can use Fedora. Fedora has different goals:

The Fedora Project is a global partnership of free software community members. The Fedora Project is sponsored by Red Hat, which invests in our infrastructure and resources to encourage collaboration and incubate innovative new technologies. Some of these technologies may later be integrated into Red Hat products. They are developed in Fedora and produced under a free and open source license from inception, so other free software communities and projects are free to study, adopt, and modify them.

There are of course tradeoffs, but you cannot have both "stable" and "new" at the same time.

| improve this answer | |
  • Thank you, Thomas. You are correct in your summary of CentOS. It is true that new cannot be guaranteed to be stable. Nevertheless, we have many users that want newer packages on CentOS systems. We are forced to offer newer versions as an option, but will follow other advice here (which is to stick within the packaging system and offer newer versions via alternate directory paths). – i_grok Apr 30 '16 at 1:50
0

Is there a consensus on what one should do here?

Yes, do not modify system-managed Python installation manually (running pip) – see What are the risks of running 'sudo pip'? For in depth discussion of the subject please read pip's issue 1668 Default to --user and other issues linked therein.

Most common solutions are:

  • Installing Python packages in user's home directory (user scheme)
  • Compiling Python from source and using this instance instead of the system one

Theoretically another possible solution would be using --target option with pip and setting PYTHONPATH environment variable. However --target option has many (currently 12) problems.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.