33

What's the recommended way of installing python packages on Arch? Searching for them on the AUR and installing them from there (or create a PKGBUILD file to make a package yourself) or using pip?

I started off by installing stuff from pacman and the AUR and don't know if it would be wise to mix with pip packages.

migrated from serverfault.com May 19 '13 at 19:12

This question came from our site for system and network administrators.

31

If you don't need the python packages for all users then you can install them in your home like this:

pip install --user packagename

Installing in your home will not conflict with the package manager.

By default pip install --user will install in your "user site" directory. Usually that is something like: /home/lesmana/.local/lib/python3.6/site-packages.

The following command will print, among others, your "user site" location:

python -m site

To customize the install location:

PYTHONUSERBASE=$HOME/some/dir pip install --user packagename

this will install everything under $HOME/some/dir

to run:

PYTHONUSERBASE=$HOME/some/dir $HOME/some/dir/bin/progname

See the pip manual for more information.


if you do want the python package for all users then the best place to install it is /opt. for example like this:

PYTHONUSERBASE=/opt/packagedir pip install packagename

(note the missing --user)

and to run, as above:

PYTHONUSERBASE=/opt/packagedir /opt/packagedir/bin/progname

Background explanation: /opt is commonly acknowledged by gnu/linux distributions as the directory where the local user or system administrator can install his own stuff. in other words: the package manager of distributions usually do not touch /opt. this is more or less standardized in the Filesystem Hierarchy Standard

For comfort for the users you will still want to write a wrapper script and place it in /bin or /usr/bin. This still bears risk of colliding with the distribution package manager but at least it is just one wrapper script file. So the damage that might be done is minimal. You can name the wrapper script something like local-foo or custom-foo to further minimize the risk of collision with the distribution package manager.

Alternatively you can modify PATH to include /opt/bin and place your wrapper script there. But this again requires you to modify a (or some) system files where PATH is defined which again may be overwritten by the distribution package manager.

In short: if you want to install for all users then do so in /opt. Where you place the wrapper script for comfort is a judgement call.

More Information about /opt and Filesystem Hierarchy Standard:

  • IMHO, this, together with a pointer to setting up virtualenvs, should be the accepted answer. – ttsiodras Jul 18 '16 at 9:05
  • 1
    Ugh. Newer to python and wish I'd known this before sudio pip-ping things. Thanks for this. – Hendy Jan 14 '17 at 17:06
  • What if you do need the python packages for all users? – Mehrdad Oct 7 at 1:17
  • @Mehrdad updated answer – lesmana Oct 7 at 10:58
  • @lesmana: Thanks! But are you sure that's the recommended way to do it? This is the first time I've heard anyone suggest using /opt for Python... – Mehrdad Oct 7 at 11:18
9

Typically, in a distribution, it's recommended that you use the distribution's package manager. You can of course install things using pip (or, in the perl world, cpan), or compile and install things yourself. However, when you do this, the distribution's package manager doesn't know about them and can't manage dependencies or updates for them.

Using pip is pretty much equivalent to compiling and installing your own package. Do it if you need to, but prefer the distribution's package manager.

  • 2
    You should absolutely avoid using pip (at least globally via sudo or as root) - I just got quite some site-package/... already exists errors when pacman tried to install some dependencies – Tobias Kienzler Oct 12 '15 at 16:55
  • I just got such errors, too. I had upgraded all pip3 packages and then pacman refused to do system upgrade due to conflicts. I had to uninstall that package both through pip3 and pacman, then do the system upgrade, and finally install the package back (using pacman, of course). – Al.G. Aug 22 '17 at 22:14
7

The right way for ArchLinux

The right way to install PYTHON packages in ArchLinux is using PACMAN! To install packages to Python3 you have to use

sudo pacman -S python-'package'

If you want to install packages from Python2, you have to use

sudo pacman -S python2-'package'

Most python packages are in the ArchLinux repositories and the packages that are not are in AUR (ArchLinux User Repositories) - for these packages you have to download the PKGBUILD file and compile. After that, you have to use PACMAN to finish the installation

makepkg -s
sudo pacman -U 'compiled-package'

The second right way for ArchLinux

When the package isn't in the AUR or the PKGBUILD isn't working, you can use PIP to install it to Python3

sudo pip install 'python-package'

or Python2

sudo pip2 install 'python-package'

You could give a chance to virtualenv or even conda

On Arch you can also use VirtualEnvironments. This can bring portability to your code and maintain old packages as well. Install it with

sudo pacman -S python-virtualenv

and try this

virtualenv -p /usr/bin/python3 yourenv
source yourenv/bin/activate
pip install package-name

When you create this environment yourenv, you will setup pip to install packages only into this environment, not to the entire system.

These other links can be useful with you want to learn more about managing packages on Linux with conda or virtualenv:

Installing Python Packages from a Jupyter Notebook

Code Python on ArchLinux

If you follow these rules, your ArchLinux will not break and won't have dependency problems between PACMAN and PIP.

Hope it's useful!

5

For certain packages (ones that I most probably don't want to hack), I make my own package using this:

https://github.com/bluepeppers/pip2arch

then build and install the PKGBUILD produced.

I leave virtualenvs for packages I might want to modify or hack.

1

In addition to the other answers here, check out the python-virtualenv package. It might be very useful if you are doing development on several projects with different dependencies with mismatching version numbers.

https://wiki.archlinux.org/index.php/Python_VirtualEnv

Also beware that there are two variants of pip and virtualenv. One for Python 2 and one for Python 3. If installation fails with a syntax error, you might be trying with the wrong version.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.