55

What's the recommended way of installing python packages on Arch? Searching for them on the AUR and installing them from there (or create a PKGBUILD file to make a package yourself) or using pip?

I started off by installing stuff from pacman and the AUR and don't know if it would be wise to mix with pip packages.

0

7 Answers 7

42

If you don't need the python packages for all users then you can install them in your home like this:

pip install --user packagename

Installing in your home will not conflict with the package manager.

By default pip install --user will install in your "user site" directory. Usually that is something like: /home/lesmana/.local/lib/python3.6/site-packages.

The following command will print, among others, your "user site" location:

python -m site

To customize the install location:

PYTHONUSERBASE=$HOME/some/dir pip install --user packagename

this will install everything under $HOME/some/dir

to run:

PYTHONUSERBASE=$HOME/some/dir $HOME/some/dir/bin/progname

See the pip manual for more information.


if you do want the python package for all users then the best place to install it is /opt. for example like this:

PYTHONUSERBASE=/opt/packagedir pip install packagename

(note the missing --user)

and to run, as above:

PYTHONUSERBASE=/opt/packagedir /opt/packagedir/bin/progname

Background explanation: /opt is commonly acknowledged by gnu/linux distributions as the directory where the local user or system administrator can install his own stuff. in other words: the package manager of distributions usually do not touch /opt. this is more or less standardized in the Filesystem Hierarchy Standard

For comfort for the users you will still want to write a wrapper script and place it in /bin or /usr/bin. This still bears risk of colliding with the distribution package manager but at least it is just one wrapper script file. So the damage that might be done is minimal. You can name the wrapper script something like local-foo or custom-foo to further minimize the risk of collision with the distribution package manager.

Alternatively you can modify PATH to include /opt/bin and place your wrapper script there. But this again requires you to modify a (or some) system files where PATH is defined which again may be overwritten by the distribution package manager.

In short: if you want to install for all users then do so in /opt. Where you place the wrapper script for comfort is a judgement call.

More Information about /opt and Filesystem Hierarchy Standard:

7
  • IMHO, this, together with a pointer to setting up virtualenvs, should be the accepted answer.
    – ttsiodras
    Commented Jul 18, 2016 at 9:05
  • 1
    Ugh. Newer to python and wish I'd known this before sudio pip-ping things. Thanks for this.
    – Hendy
    Commented Jan 14, 2017 at 17:06
  • What if you do need the python packages for all users?
    – user541686
    Commented Oct 7, 2019 at 1:17
  • 1
    well it is my recommendation at least. mainly based on what I read about FHS and my desire to not conflict with distribution package manager. also i see no reason at all to not install python (or any other language) in /opt. if a package does not work if installed in /opt then it most likely has hardcoded paths which i consider a bug.
    – Lesmana
    Commented Oct 7, 2019 at 19:50
  • 2
    The --user is forbidden (you get a printout about an "externally-managed-environment" and how to use virtualenv if the package isn't in Arch). If it's not available as an Arch package, virtualenv looks to be the only recourse. Commented Aug 14, 2023 at 19:13
28

The right way for ArchLinux

The right way to install PYTHON packages in ArchLinux is using PACMAN! To install packages to Python3 you have to use

sudo pacman -S python-'package'

If you want to install packages from Python2, you have to use

sudo pacman -S python2-'package'

Most python packages are in the ArchLinux repositories and the packages that are not in AUR (ArchLinux User Repositories) - for these packages you have to download the PKGBUILD file and compile. After that, you have to use PACMAN to finish the installation

makepkg -s
sudo pacman -U 'compiled-package'

The second right way for ArchLinux

When the package isn't in the AUR or the PKGBUILD isn't working, you can use PIP to install it to Python3

sudo pip install 'python-package'

or Python2

sudo pip2 install 'python-package'

BE AWARE: when you are using pip the same installation folder is shared with pacman and most of time, especially when you are updating all system packages (sudo pacman -Suy), will raise a conflict error. You always have to prefer the first option above.To solve conflict problems, just uninstall pip package and install they equivalent package on pacman (pip uninstall 'python-package').

You could give a chance to virtualenv or even conda

If you are planning to develop some python application or python package your better option is use virtual environments.

For python packaging applications you should try poetry it is current better option to manage application from start to finish. It is a much better option than requirements.txt + setup.py.

Another more simple option is use python-virtualenv. This can bring portability to your code and maintain old packages as well. Install it with

sudo pacman -S python-virtualenv

and try this

virtualenv -p /usr/bin/python3 yourenv
source yourenv/bin/activate
pip install package-name

When you create this environment yourenv, you will setup pip to install packages only into this environment, not to the entire system.

These other links can be useful with you want to learn more about managing packages on Linux with conda or virtualenv:

Installing Python Packages from a Jupyter Notebook

If you follow these rules, your ArchLinux will not break and won't have dependency problems between PACMAN and PIP.

Hope it's useful!

5
  • I read somewhere else that sudo pip install might conflict with pacman.
    – Student
    Commented Jul 4, 2020 at 19:54
  • 1
    Yes, pacman is always conflicting with pip cause sudo pip command wants to install in the same directory of pacman. If you want to use a closed environment to work with python packages as developer, you can use poetry it works very well to manager and packing python applications. Nowadays, I'm using virtualenvwrapper to manager my python environments along side to pacman. Commented Jul 9, 2020 at 16:32
  • I generally agree with your conclusions, but as for me, you miss some explanation about your reasons (I like Falcon Momot's answer more). However, the article you link to "Code Python on ArchLinux" is pure garbage (hope no one here is offended) (mark its beginning: "ArchLinux is fantastic,because you can use Python3 as your default Python version" - pure nonsense) and even contradicts your post (because it doesn't even mention pacman), therefore I have to downvote. UPD: maybe not downvote. Commented Jun 23, 2022 at 12:08
  • @YaroslavNikitenko I agree with you, the article is so lame that I've removed from my post (4 years ago I've trying to use virtualenv and that post has some guidelines). BTW, right now I've working with poetry to packaging and deploy Python applications. Commented Jun 24, 2022 at 13:15
  • Thanks for the fix! I think I shall upvote your answer after that. For my projects setuptools are more than enough (and I prefer more official tools). Commented Jun 24, 2022 at 14:05
12

Update in 2021 for current practice:

If you are trying to ship software as part of the distribution, use the distribution's package manager. If you're doing anything else, the answer is most likely to use a venv (see https://docs.python.org/3/tutorial/venv.html).

This will help you with cases where your distribution ships old packages, or has some packages that depend on an old version of a package, and helps you avoid the impact of system upgrades and incompatibilities between versions of python.

Avoid using pip outside a venv, and especially avoid using pip as root. That will just clobber system packages and break things. The awkwardness between the language having a package manager that can be system-wide and the distribution's need to encapsulate package dependency management interact poorly in practice, but a venv allows you to escape that.


Answer from 2013:

Typically, in a distribution, it's recommended that you use the distribution's package manager. You can of course install things using pip (or, in the perl world, cpan), or compile and install things yourself. However, when you do this, the distribution's package manager doesn't know about them and can't manage dependencies or updates for them.

Using pip is pretty much equivalent to compiling and installing your own package. Do it if you need to, but prefer the distribution's package manager.

2
  • 3
    You should absolutely avoid using pip (at least globally via sudo or as root) - I just got quite some site-package/... already exists errors when pacman tried to install some dependencies Commented Oct 12, 2015 at 16:55
  • I just got such errors, too. I had upgraded all pip3 packages and then pacman refused to do system upgrade due to conflicts. I had to uninstall that package both through pip3 and pacman, then do the system upgrade, and finally install the package back (using pacman, of course).
    – Al.G.
    Commented Aug 22, 2017 at 22:14
6

For certain packages (ones that I most probably don't want to hack), I make my own package using this:

https://github.com/bluepeppers/pip2arch

then build and install the PKGBUILD produced.

I leave virtualenvs for packages I might want to modify or hack.

2
2

In addition to the other answers here, check out the python-virtualenv package. It might be very useful if you are doing development on several projects with different dependencies with mismatching version numbers.

https://wiki.archlinux.org/index.php/Python_VirtualEnv

Also beware that there are two variants of pip and virtualenv. One for Python 2 and one for Python 3. If installation fails with a syntax error, you might be trying with the wrong version.

2
  • In 2023 the Arch recommended way to install non-Arch-packaged python modules is to use pipx.

  • Trying to install a system wide module with pip now displays a warning showing a --break-system-packages flag being required.

  • pipx creates self contained installs in their own venv: ~/.local/pipx/venvs/package_name

2

If you get This environment is externally managed... when you attempt a pip install in archlinux/manjaro/ubuntu/debian/other distros, then the supposedly UNRECOMMENDED way (assuming version 3.11) is:

sudo rm /usr/lib/python3.11/EXTERNALLY-MANAGED

Feel free to downvote the answer... but you are in charge of your system and IT IS YOU who decides whether your pip packages should be 'externally managed', what belongs in a virtualenv and what can be global.

1
  • 1
    thanks, pip install --user is back.
    – cronos
    Commented Nov 21, 2023 at 11:20

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .