0

Have a problem with some nodes at a remote site where several Python RPMs were installed on SLES systems where pip had previously been used. Although we have given guidance to the remote teams to not mix the two, and they now follow this guidance, there were still some machines that were 'mixed'.

On some, but not all, of the machines at the sites, running 'pip list' shows that pip has become aware of the newer packages. Pip was updated during this process (via RPM), but it was installed in the same order as the other Python packages on all nodes without having the same behavior. As a result of this, some of our scripts now fail to run properly due to incompatibilities caused by the newer packages.

There are certainly going to be some differences between the nodes at this site, but in general, they are kept as pretty close to identical as possible. Does anyone have any insight into how a yum install of Python packages can update the pip database?

1 Answer 1

2

Does anyone have any insight into how a yum install of Python packages can update the pip database?

For this purpose, pip doesn't really have a database separate to that of Python itself. When packages are installed, an associated <package>.egg-info is placed next to the package itself. This is core-python and not pip or yum. So when you ask pip what's there it will just look for these egg-info meta files and use those to inform it what is already installed. eg: /usr/lib/python3/dist-packages on ubuntu.

When you ask yum to install a python package, the RPM should ship with these same egg-info files.

Part of Pip's job is checking dependencies in the python environment when you install. If yum has broken dependencies then this will be discovered (as you've found out). It does this by checking the meta data in the egg-info files.

Yum also tries to check dependencies. But there's a big catch. Yum isn't actually aware at all of what python libraries are actually installed. It only knows what RPMs are installed. So any python packages installed without using yum or otherwise an RPM, will not be discovered by yum.


This is really what virtual environments were created for. It's often not a good idea to bind your application environment to your system environment. There's rarely a good way to keep your application requirements in-line with your OS.


Under what circumstances can pip 'discover' package updates made by yum

If yum has installed packages into the same environment then pip should be able to discover them. The two major exceptions might be:

  • If you were installing the packages into a virtual environment. Unless specificly requested, virtual environments don't inherit system packages. Even when the environment does inherit them, it will always use it's own local install first.

  • If pip installed the packages to the user instead of the system. Again these would not be overwritten by yum and python would typically use packages installed for the User (under /home/... over packages installed for the system.

The second of these is quite likely to be a source of difference between machines. pip install might install to the user where sudo pip install might install for the system.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .