Repairing python setup

Question

On my Kubuntu 14.4 (which has python 2.7.6 as standard) my python is broken after I tried to install python 2.7.10 after building from source from python.org with the help of How to install the latest Python version on Debian separately or upgrade?. I am not able to repair it with the standard commands I suspect that my dpkg is somehow confused/broken regarding the python installation. I would like to fix dpkg in this aspect. I suspect that this has something to do with the file /var/lib/dpkg/status and /var/lib/dpkg/available and /var/lib/dpkg/info/* particularily the first. I think I have to reset dpkg somehow, but I am really no expert. The reason why I think this is:

$ apt-cache policy python
python:
  Installed: 2.7.10-1
  Candidate: 2.7.10-1
  Version table:
*** 2.7.10-1 0
    100 /var/lib/dpkg/status
2.7.5-5ubuntu3 0
    500 http://de.archive.ubuntu.com/ubuntu/ trusty/main amd64 Packages

$ /usr/bin/python2.7
Python 2.7.6 (default, Jun 22 2015, 17:58:13) 
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> exit()

The Reason I tried to install python 2.7.10 is because I needed it for another program (because of issues with ssl / openssl of python 2.7.6), but now I just want to get my system repaired - just let it be python 2.7.6.

---

The Full Technical

I started trying to solve this by asking on ubuntu https://askubuntu.com/questions/648424/muon-is-gone-after-change-of-python-issues-after-python-2-7-10-installation-on but I did not get any answer there. Maybe it was the wrong crowd. I have tried quite a bit since then and have an idea what's the problem, but don't know the steps to accomplish this.

It started with me not being able to install muon with sudo apg-get install muon:

$ sudo apt-get install muon
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 muon : Depends: apt-xapian-index but it is not going to be installed
E: Unable to correct problems, you have held broken packages.

The typical advice (e.g. from https://askubuntu.com/questions/118749/package-system-is-broken-how-to-fix-it) does not help:

sudo apt-get autoremove
sudo apt-get clean
sudo apt-get autoclean
sudo apt-get update
sudo apt-get upgrade -f
sudo apt-get -f install muon

or

sudo apt-get -f install

or

sudo dpkg --configure -a 
sudo apt-get update && sudo apt-get dist-upgrade
sudo apt-get install muon

or

sudo apt-get -o dpkg::options::="--force-confnew" -o dpkg::options::="--force-confmiss" --reinstall install muon

did not help. So I tried

$ sudo apt-get install apt-xapian-index
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 apt-xapian-index : Depends: python-xapian (>= 1.0.2) but it is not going to be installed
                    Depends: python-apt (>= 0.7.93.2) but it is not going to be installed
                    Depends: python-debian (>= 0.1.14) but it is not going to be installed
                    Depends: python:any (>= 2.7.1-0ubuntu2)
E: Unable to correct problems, you have held broken packages.

and found out the issue is with other programs as well like

$ sudo apt-get install meld 
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 meld : Depends: python:any (>= 2.7.1-0ubuntu2)
        Depends: python-gtk2 (>= 2.14) but it is not going to be installed
        Depends: python-glade2 (>= 2.14) but it is not going to be installed
        Depends: python-gobject-2 (>= 2.16) but it is not going to be installed
        Recommends: python-gnome2 but it is not going to be installed
        Recommends: python-gconf but it is not going to be installed
        Recommends: python-gtksourceview2 (>= 2.4) but it is not going to be installed
E: Unable to correct problems, you have held broken packages.

So I tried (without luck)

$ sudo update-alternatives --config python
update-alternatives: error: no alternatives for python

The following did not help either:

sudo dpkg -P python2.7
sudo apt-get install python2.7
sudo dpkg -P python-minimal
sudo apt-get autoremove && sudo apt-get clean sudo apt-get update && sudo apt-get -f install

I am getting

$ apt-cache policy python
python:
  Installed: 2.7.10-1
  Candidate: 2.7.10-1
  Version table:
*** 2.7.10-1 0
    100 /var/lib/dpkg/status
2.7.5-5ubuntu3 0
    500 http://de.archive.ubuntu.com/ubuntu/ trusty/main amd64 Packages

Trying to reinstall python does not work

$ sudo apt-get -o dpkg::options::="--force-confnew" -o dpkg::options::="--force-confmiss" --reinstall install python
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Reinstallation of python is not possible, it cannot be downloaded.
0 upgraded, 0 newly installed, 0 to remove and 16 not upgraded.

or

$ sudo apt-get -o dpkg::options::="--force-confnew" -o dpkg::options::="--force-confmiss" --reinstall install python2
Reading package lists... Done
Building dependency tree       
Reading state information... Done
E: Unable to locate package python2

and trying to build an uninstaller does not work either:

~/Python-2.7.10$ sudo make uninstall 
make: *** No rule to make target `uninstall'.  Stop.

So I started to suspect that I have to get dpkg fixed somehow, because

$ apt-cache policy python
python:
  Installed: 2.7.10-1
  Candidate: 2.7.10-1
  Version table:
*** 2.7.10-1 0
    100 /var/lib/dpkg/status
2.7.5-5ubuntu3 0
    500 http://de.archive.ubuntu.com/ubuntu/ trusty/main amd64 Packages

$ /usr/bin/python2.7
Python 2.7.6 (default, Jun 22 2015, 17:58:13) 
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> exit()

---

More information (Appendix)

$ dpkg -l python* | grep -v ^un
Gewünscht=Unbekannt/Installieren/R=Entfernen/P=Vollständig Löschen/Halten
| Status=Nicht/Installiert/Config/U=Entpackt/halb konFiguriert/
    Halb installiert/Trigger erWartet/Trigger anhängig
|/ Fehler?=(kein)/R=Neuinstallation notwendig (Status, Fehler: GROSS=schlecht)
||/ Name                                        Version                                 Architektur  Beschreibung
+++-===========================================-=======================================-============-=====================================================================================================================================================================================================================
ii  python                                      2.7.10-1                                amd64        Python 2.7.10
ii  python-apt-common                           0.9.3.5ubuntu1                          all          Python interface to libapt-pkg (locales)
ii  python-chardet-whl                          2.2.1-2~ubuntu1                         all          universal character encoding detector
ii  python-colorama-whl                         0.2.5-0.1ubuntu2                        all          Cross-platform colored terminal text in Python - Wheels
ii  python-cups                                 1.9.66-0ubuntu2                         amd64        Python bindings for CUPS
rc  python-cupshelpers                          1.4.3+20140219-0ubuntu2.6               all          Python modules for printer configuration with CUPS
ii  python-dbus-dev                             1.2.0-2build2                           all          main loop integration development files for python-dbus
ii  python-distlib-whl                          0.1.8-1ubuntu1                          all          low-level components of python distutils2/packaging
rc  python-gobject-2                            2.28.6-12build1                         amd64        deprecated static Python bindings for the GObject library
ii  python-html5lib-whl                         0.999-3~ubuntu1                         all          HTML parser/tokenizer based on the WHATWG HTML5 specification
ii  python-ldb                                  1:1.1.16-1                              amd64        Python bindings for LDB
ii  python-minimal                              2.7.5-5ubuntu3                          amd64        minimal subset of the Python language (default version)
ii  python-ntdb                                 1.0-2ubuntu1                            amd64        Python bindings for NTDB
ii  python-pam                                  0.4.2-13.1ubuntu3                       amd64        Python interface to the PAM library
ii  python-pip-whl                              1.5.4-1ubuntu3                          all          alternative Python package installer
ii  python-renderpm                             3.0-1build1                             amd64        python low level render interface
ii  python-reportlab-accel                      3.0-1build1                             amd64        C coded extension accelerator for the ReportLab Toolkit
ii  python-requests-whl                         2.2.1-1ubuntu0.3                        all          elegant and simple HTTP library for Python, built for human beings
ii  python-setuptools-whl                       3.3-1ubuntu2                            all          Python Distutils Enhancements (wheel package)
ii  python-six-whl                              1.5.2-1ubuntu1                          all          Python 2 and 3 compatibility library (universal wheel)                                               
rc  python-support                              1.0.15                                  all          automated rebuilding support for Python modules                                                      
ii  python-talloc                               2.1.0-1                                 amd64        hierarchical pool based memory allocator - Python bindings                                           
ii  python-tdb                                  1.2.12-1                                amd64        Python bindings for TDB                                                                              
ii  python-twisted-bin                          13.2.0-1ubuntu1                         amd64        Event-based framework for internet applications                                                      
rc  python-twisted-core                         13.2.0-1ubuntu1                         all          Event-based framework for internet applications                                                      
rc  python-ubuntu-sso-client                    13.10-0ubuntu6                          all          Ubuntu Single Sign-On client - Python library                                                        
ii  python-urllib3-whl                          1.7.1-1ubuntu3                          all          HTTP library with thread-safe connection pooling                                                     
ii  python2.7                                   2.7.6-8ubuntu0.2                        amd64        Interactive high-level object-oriented language (version 2.7)                                        
ii  python2.7-minimal                           2.7.6-8ubuntu0.2                        amd64        Minimal subset of the Python language (version 2.7)                                                  
ii  python3                                     3.4.0-0ubuntu2                          amd64        interactive high-level object-oriented language (default python3 version)                            
ii  python3-apport                              2.14.1-0ubuntu3.11                      all          Python 3 library for Apport crash report handling                                                    
ii  python3-apt                                 0.9.3.5ubuntu1                          amd64        Python 3 interface to libapt-pkg                                                                     
ii  python3-aptdaemon                           1.1.1-1ubuntu5.2                        all          Python 3 module for the server and client of aptdaemon                                               
ii  python3-chardet                             2.2.1-2~ubuntu1                         all          universal character encoding detector for Python3                                                    
ii  python3-colorama                            0.2.5-0.1ubuntu2                        all          Cross-platform colored terminal text in Python - Python 3.x                                          
ii  python3-commandnotfound                     0.3ubuntu12                             all          Python 3 bindings for command-not-found.                                                             
ii  python3-dbus                                1.2.0-2build2                           amd64        simple interprocess messaging system (Python 3 interface)                                            
ii  python3-dbus.mainloop.qt                    4.10.4+dfsg-1ubuntu1                    amd64        D-Bus Support for PyQt4 with Python 3
ii  python3-debian                              0.1.21+nmu2ubuntu2                      all          Python 3 modules to work with Debian-related data formats
ii  python3-defer                               1.0.6-2build1                           all          Small framework for asynchronous programming (Python 3)
ii  python3-dev                                 3.4.0-0ubuntu2                          amd64        header files and a static library for Python (default)
ii  python3-distlib                             0.1.8-1ubuntu1                          all          low-level components of python distutils2/packaging
ii  python3-distupgrade                         1:0.220.7                               all          manage release upgrades
ii  python3-gdbm:amd64                          3.4.0-0ubuntu1                          amd64        GNU dbm database support for Python 3.x
ii  python3-gi                                  3.12.0-1ubuntu1                         amd64        Python 3 bindings for gobject-introspection libraries
ii  python3-html5lib                            0.999-3~ubuntu1                         all          HTML parser/tokenizer based on the WHATWG HTML5 specification (Python 3)
ii  python3-minimal                             3.4.0-0ubuntu2                          amd64        minimal subset of the Python language (default python3 version)
ii  python3-pip                                 1.5.4-1ubuntu3                          all          alternative Python package installer - Python 3 version of the package
ii  python3-pkg-resources                       3.3-1ubuntu2                            all          Package Discovery and Resource Access using pkg_resources
ii  python3-problem-report                      2.14.1-0ubuntu3.11                      all          Python 3 library to handle problem reports
ii  python3-pycurl                              7.19.3-0ubuntu3                         amd64        Python 3 bindings to libcurl
ii  python3-pykde4                              4:4.13.3-0ubuntu0.1                     amd64        Python 3 bindings for the KDE Development Platform
ii  python3-pyqt4                               4.10.4+dfsg-1ubuntu1                    amd64        Python3 bindings for Qt4
ii  python3-requests                            2.2.1-1ubuntu0.3                        all          elegant and simple HTTP library for Python3, built for human beings
ii  python3-setuptools                          3.3-1ubuntu2                            all          Python3 Distutils Enhancements
ii  python3-sip                                 4.15.5-1build1                          amd64        Python 3/C++ bindings generator runtime library
ii  python3-six                                 1.5.2-1ubuntu1                          all          Python 2 and 3 compatibility library (Python 3 interface)
ii  python3-software-properties                 0.92.37.3                               all          manage the repositories that you install software from
ii  python3-uno                                 1:4.2.8-0ubuntu2                        amd64        Python-UNO bridge
ii  python3-update-manager                      1:0.196.13                              all          python 3.x module for update-manager
ii  python3-urllib3                             1.7.1-1ubuntu3                          all          HTTP library with thread-safe connection pooling for Python3
ii  python3-wheel                               0.24.0-1~ubuntu1                        all          built-package format for Python
ii  python3-xkit                                0.5.0ubuntu2                            all          library for the manipulation of xorg.conf files (Python 3)
ii  python3.4                                   3.4.0-2ubuntu1.1                        amd64        Interactive high-level object-oriented language (version 3.4)
ii  python3.4-dev                               3.4.0-2ubuntu1.1                        amd64        Header files and a static library for Python (v3.4)
ii  python3.4-minimal                           3.4.0-2ubuntu1.1                        amd64        Minimal subset of the Python language (version 3.4)

---

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 14.04.2 LTS
Release:        14.04
Codename:       trusty

---

$ grep -P '^[ \t]*[^#[ \t]+' /etc/apt/sources.list /etc/apt/sources.list.d/*.list
/etc/apt/sources.list:deb http://de.archive.ubuntu.com/ubuntu/ trusty main restricted
/etc/apt/sources.list:deb-src http://de.archive.ubuntu.com/ubuntu/ trusty main restricted
/etc/apt/sources.list:deb http://de.archive.ubuntu.com/ubuntu/ trusty-updates main restricted
/etc/apt/sources.list:deb-src http://de.archive.ubuntu.com/ubuntu/ trusty-updates main restricted
/etc/apt/sources.list:deb http://de.archive.ubuntu.com/ubuntu/ trusty universe
/etc/apt/sources.list:deb-src http://de.archive.ubuntu.com/ubuntu/ trusty universe
/etc/apt/sources.list:deb http://de.archive.ubuntu.com/ubuntu/ trusty-updates universe
/etc/apt/sources.list:deb-src http://de.archive.ubuntu.com/ubuntu/ trusty-updates universe
/etc/apt/sources.list:deb http://de.archive.ubuntu.com/ubuntu/ trusty multiverse
/etc/apt/sources.list:deb-src http://de.archive.ubuntu.com/ubuntu/ trusty multiverse
/etc/apt/sources.list:deb http://de.archive.ubuntu.com/ubuntu/ trusty-updates multiverse
/etc/apt/sources.list:deb-src http://de.archive.ubuntu.com/ubuntu/ trusty-updates multiverse
/etc/apt/sources.list:deb http://de.archive.ubuntu.com/ubuntu/ trusty-backports main restricted universe multiverse
/etc/apt/sources.list:deb-src http://de.archive.ubuntu.com/ubuntu/ trusty-backports main restricted universe multiverse
/etc/apt/sources.list:deb http://security.ubuntu.com/ubuntu trusty-security main restricted
/etc/apt/sources.list:deb-src http://security.ubuntu.com/ubuntu trusty-security main restricted
/etc/apt/sources.list:deb http://security.ubuntu.com/ubuntu trusty-security universe
/etc/apt/sources.list:deb-src http://security.ubuntu.com/ubuntu trusty-security universe
/etc/apt/sources.list:deb http://security.ubuntu.com/ubuntu trusty-security multiverse
/etc/apt/sources.list:deb-src http://security.ubuntu.com/ubuntu trusty-security multiverse
/etc/apt/sources.list:deb http://archive.canonical.com/ubuntu trusty partner
/etc/apt/sources.list:deb http://extras.ubuntu.com/ubuntu trusty main
/etc/apt/sources.list:deb http://cran.uni-muenster.de/bin/linux/ubuntu trusty/
/etc/apt/sources.list.d/fossfreedom-packagefixes-trusty.list:deb http://ppa.launchpad.net/fossfreedom/packagefixes/ubuntu trusty main
/etc/apt/sources.list.d/jitsi.list:deb http://download.jitsi.org/deb unstable/
/etc/apt/sources.list.d/leviatan1-ppa-trusty.list:deb http://ppa.launchpad.net/leviatan1/ppa/ubuntu trusty main

---

$ whereis python
python: /usr/bin/python /usr/bin/python3.4-config /usr/bin/python3.4 /usr/bin/python3.4m /usr/bin/python2.7 /usr/bin/python3.4m-config /etc/python /etc/python3.4 /etc/python2.7 /usr/lib/python3.4 /usr/lib/python2.7 /usr/bin/X11/python /usr/bin/X11/python3.4-config /usr/bin/X11/python3.4 /usr/bin/X11/python3.4m /usr/bin/X11/python2.7 /usr/bin/X11/python3.4m-config /usr/local/lib/python3.4 /usr/local/lib/python2.7 /usr/include/python3.4 /usr/include/python3.4m /usr/share/python /usr/share/man/man1/python.1.gz

---

$ whereis python2.7
python2: /usr/bin/python2.7 /usr/bin/python2 /etc/python2.7 /usr/lib/python2.7 /usr/bin/X11/python2.7 /usr/bin/X11/python2 /usr/local/lib/python2.7 /usr/share/man/man1/python2.1.gz

Answer 1

You have installed Python packages that are more recent than what your distribution provides. For example, you have python version 2.7.10-1 installed but your distribution only has version 2.7.5-5ubuntu3.

APT doesn't downgrade packages unless explicitly told to do so. So for example if you try to install a package that depends on the exact version of Python, it won't work, because the python package can't be downgraded. Even apt-get --reinstall install python fails because APT won't downgrade Python to 2.7.5.

In order to repair your system, you need to allow APT to perform downgrades. To do that, define APT preferences. Create a file /etc/apt/preferences.d/allow-downgrade containing

Package: *
Pin: release o=Ubuntu
Pin-Priority: 1001

The files in /etc/apt/preferences.d (plus /etc/apt/preferences) contain priority declarations that override the default selection when multiple versions of a package are available, which is “prefer the latest version from the target distribution”. Giving a package a priority over 1000 causes it to be preferred even if it's an older version that a package with a lower priority. Installed packages have priority 500 so the package from Ubuntu wins. For more information see:

man apt_preferences

I think once you've set these priorities you can run

apt-get update
apt-get upgrade

to downgrade all your packages to the version in Ubuntu (packages not in Ubuntu won't be removed). Also run apt-get -f install and don't try to install any other software until this completes successfully.

Once everything is downgraded, remove the preferences file and run apt-get update again.

Answer 2

There is only one way I discovered that was working.

1. Step 1: download python 2.7 deb package from here and install it

   sudo dpkg -i <package-name>
   

2. Use dbkg tool to list everything related to python

   sudo dpkg -l | grep python
   

3. remove everything found above

   sudo dpkg --purge --force-all package name

4. try to do

   sudo apt upgrade
   

it will give you a list of packages at the end that corrupt. Remove them using step 3

5. Repeat steps 3-4 until sudo apt upgrade does not give any errors

6. You might need to create a symlink to /usr/bin/python using location of python 2.7 you installed as deb package. (locate bin/python) and ln -s <path> /usr/bin/python

7. At this moment python should be fixed. But do-release-upgrade (read manual for this how to update to the latest ubuntu, not LTS)