I'm getting a python unicode error every time I try to install or update anything with apt or aptitude, preventing me to update my system.

The error I'm getting is the following:

File "/usr/bin/py3clean", line 4
SyntaxError: Non-ASCII character '\xc2' in file /usr/bin/py3clean on line 4, but no encoding declared; see http://python.org/dev/peps/pep-0263/ for details

I follow the link and it talks about magic comments to set python encoding, but of course it is not the way to go. I can bypass the error in py3clean manually adding magic comments, but then another python file reports the problem.

I'm not a python user so I'm quite stuck with it... Is there any system wide var I can set to control python encoding or any other method to solve it?

I'm with Debian testing 3.16.0-4-686-pae #1 SMP Debian 3.16.7-ckt4-3 (2015-02-03) i686 GNU/Linux.


Edit: Following is the output of aptitude dist-upgrade

The following packages will be upgraded: 
  apache2 apache2-bin apache2-data apache2-dev apache2-utils apache2.2-bin apt apt-utils avahi-daemon bind9-host binutils bsd-mailx 
  bsdutils busybox bzip2 cli-common comerr-dev console-setup console-setup-linux coreutils cpio cpp cpp-4.8 cpp-4.9 cups cups-browsed 
  cups-bsd cups-client cups-common cups-core-drivers cups-daemon cups-filters cups-filters-core-drivers cups-ppdc cups-server-common 
  curl dbus dbus-x11 debconf debconf-i18n debhelper dmsetup dnsmasq-base dnsutils dpkg dpkg-dev dselect e2fslibs e2fsprogs evince 
  evince-common evolution-data-server evolution-data-server-common exim4 exim4-base exim4-config exim4-daemon-light file 
  foomatic-db-compressed-ppds freerdp-x11 g++ g++-4.9 gcc gcc-4.8 gcc-4.8-base gcc-4.9 gcc-4.9-base gdisk gdm3 geoip-database 
  gir1.2-ebookcontacts-1.2 gir1.2-edataserver-1.2 gir1.2-evince-3.0 gir1.2-gdm3 gir1.2-gmenu-3.0 gir1.2-gtk-2.0 
  gir1.2-javascriptcoregtk-3.0 gir1.2-networkmanager-1.0 gir1.2-rest-0.7 gir1.2-upowerglib-1.0 gir1.2-webkit-3.0 gnome-font-viewer 
  gnome-orca gnome-settings-daemon gnupg gnupg-agent gnupg2 google-chrome-stable gpgv growisofs grub-common grub-pc grub-pc-bin 
  grub2-common gstreamer1.0-plugins-bad gstreamer1.0-plugins-ugly gtk2-engines-pixbuf hdparm heroku heroku-toolbelt hipchat host 
  icedtea-7-jre-jamvm icedtea-netx icedtea-netx-common iceweasel iceweasel-l10n-ca icu-devtools ifupdown initramfs-tools initscripts 
  installation-report isc-dhcp-client isc-dhcp-common keyboard-configuration krb5-locales krb5-multidev lftp libapache2-mod-php5 
  libapt-inst1.5 libapt-pkg4.12 libarchive13 libasan0 libasan1 libatomic1 libav-tools libavahi-client-dev libavahi-client3 
  libavahi-common-data libavahi-common-dev libavahi-common3 libavahi-core7 libavahi-glib-dev libavahi-glib1 libavahi-gobject0 
  libavahi-ui-gtk3-0 libavcodec56 libavdevice55 libavfilter5 libavformat56 libavresample2 libavutil54 libbind9-90 libblkid1 
  libbrlapi0.6 libbz2-1.0 libbz2-dev libc-bin libc-dev-bin libc6 libc6-dev libc6-i686 libcamel-1.2-49 libcap2-bin libcilkrts5 
  libcomerr2 libcups2 libcupscgi1 libcupsfilters1 libcupsimage2 libcupsmime1 libcupsppdc1 libcurl3 libcurl3-gnutls libcurl4-openssl-dev 
  libdb5.3 libdb5.3-dev libdb5.3-java libdb5.3-java-jni libdbus-1-3 libdbus-1-dev libdevmapper-event1.02.1 libdevmapper1.02.1 
  libdns-export100 libdns100 libdpkg-perl libebackend-1.2-7 libebook-1.2-14 libebook-contacts-1.2-0 libecal-1.2-16 libedata-book-1.2-20 
  libedata-cal-1.2-23 libedataserver-1.2-18 libelf1 libevdocument3-4 libevview3-3 libfontembed1 libfreerdp-cache1.1 
  libfreerdp-client1.1 libfreerdp-codec1.1 libfreerdp-common1.1.0 libfreerdp-core1.1 libfreerdp-crypto1.1 libfreerdp-gdi1.1 
  libfreerdp-locale1.1 libfreerdp-plugins-standard libfreerdp-primitives1.1 libfreerdp-rail1.1 libfreerdp-utils1.1 libfreetype6 
  libfreetype6-dev libgail-common libgail-dev libgail18 libgcc-4.8-dev libgcc-4.9-dev libgcc1 libgcrypt20 libgdm1 libgfortran3 
  libgit2-21 libglib2.0-cil libgnome-menu-3-0 libgnutls-deb0-28 libgnutls-openssl27 libgnutls28-dev libgnutlsxx28 libgomp1 
  libgssapi-krb5-2 libgssrpc4 libgstreamer-plugins-bad1.0-0 libgtk2-perl libgtk2.0-0 libgtk2.0-bin libgtk2.0-cil libgtk2.0-common 
  libgtk2.0-dev libgudev-1.0-0 libicu-dev libicu52 libirs-export91 libisc-export95 libisc95 libisccc90 libisccfg-export90 libisccfg90 
  libitm1 libjavascriptcoregtk-1.0-0 libjavascriptcoregtk-3.0-0 libjpeg62-turbo libk5crypto3 libkadm5clnt-mit9 libkadm5srv-mit9 
  libkdb5-7 libkrb5-3 libkrb5-dev libkrb5support0 libldap-2.4-2 libldap2-dev libldb1 libllvm3.5 liblvm2app2.2 liblwres90 libmagic-dev 
  libmagic1 libmono-cairo4.0-cil libmono-corlib4.0-cil libmono-corlib4.5-cil libmono-i18n-west4.0-cil libmono-i18n4.0-cil 
  libmono-posix4.0-cil libmono-security4.0-cil libmono-sharpzip4.84-cil libmono-system-configuration4.0-cil libmono-system-core4.0-cil 
  libmono-system-drawing4.0-cil libmono-system-security4.0-cil libmono-system-xml4.0-cil libmono-system4.0-cil libmount1 libmp3lame0 
  libmpeg2-4 libmysqlclient-dev libmysqlclient18 libmysqld-dev libnm-glib-vpn1 libnm-glib4 libnm-util2 libobjc4 libpam-cap 
  libpam-systemd libparted2 libperl5.20 libpotrace0 libprocps3 libpython-dev libpython-stdlib libpython2.7 libpython2.7-dev 
  libpython2.7-minimal libpython2.7-stdlib libqt4-core libqt4-dbus libqt4-declarative libqt4-designer libqt4-dev libqt4-dev-bin 
  libqt4-help libqt4-network libqt4-opengl libqt4-opengl-dev libqt4-qt3support libqt4-script libqt4-scripttools libqt4-sql 
  libqt4-sql-mysql libqt4-svg libqt4-test libqt4-xml libqt4-xmlpatterns libqt5webkit5 libqtcore4 libqtdbus4 libqtgui4 libquadmath0 
  librest-0.7-0 librtmp1 libruby2.1 librygel-core-2.4-2 librygel-renderer-2.4-2 librygel-renderer-gst-2.4-2 librygel-server-2.4-2 
  libsasl2-2 libsasl2-dev libsasl2-modules libsasl2-modules-db libsmartcols1 libsmbclient libsnmp-base libsnmp-dev libsnmp30 libss2 
  libssh2-1 libssl-dev libssl-doc libssl1.0.0 libstdc++-4.8-dev libstdc++-4.9-dev libstdc++6 libsvn-perl libsvn1 libswscale3 
  libsystemd-daemon-dev libsystemd-dev libsystemd0 libtasn1-6 libtasn1-6-dev libtasn1-doc libtbb2 libtcl8.6 libteamdctl0 libtiff5 
  libubsan0 libudev1 libupower-glib3 libuuid1 libv8-3.14-dev libv8-3.14.5 libwbclient0 libwebkitgtk-1.0-0 libwebkitgtk-1.0-common 
  libwebkitgtk-3.0-0 libwebkitgtk-3.0-common libwinpr-crt0.1 libwinpr-crypto0.1 libwinpr-dsparse0.1 libwinpr-environment0.1 
  libwinpr-file0.1 libwinpr-handle0.1 libwinpr-heap0.1 libwinpr-input0.1 libwinpr-interlocked0.1 libwinpr-library0.1 libwinpr-path0.1 
  libwinpr-pool0.1 libwinpr-registry0.1 libwinpr-rpc0.1 libwinpr-sspi0.1 libwinpr-synch0.1 libwinpr-sysinfo0.1 libwinpr-thread0.1 
  libwinpr-utils0.1 libxfont1 libxfreerdp-client1.1 libxml2 libxml2-dev libxml2-utils lintian linux-compiler-gcc-4.8-x86 
  linux-headers-3.16.0-4-686-pae linux-headers-3.16.0-4-common linux-image-3.16.0-4-686-pae locales locales-all mongodb mongodb-clients 
  mongodb-dev mongodb-server mono-4.0-gac mono-gac mono-runtime mono-runtime-common mono-runtime-sgen mount multiarch-support 
  mysql-client mysql-client-5.5 mysql-common mysql-server mysql-server-5.5 mysql-server-core-5.5 network-manager nodejs nodejs-dev 
  openjdk-7-jdk openjdk-7-jre openjdk-7-jre-headless openprinting-ppds openssh-client openssl parted patch perl perl-base perl-modules 
  php-pear php5 php5-cli php5-common php5-dev php5-intl php5-mcrypt php5-mysql php5-readline postgresql-9.4 ppp procmail procps python 
  python-brlapi python-debian python-dev python-imaging python-ldb python-libxml2 python-minimal python-pil python-pygments 
  python-samba python-xapian python2.7 python2.7-dev python2.7-minimal python3-brlapi python3-debian qdbus qt4-linguist-tools qt4-qmake 
  qtcore4-l10n rsync rtmpdump ruby ruby-dev ruby2.1 ruby2.1-dev rygel rygel-playbin rygel-preferences rygel-tracker samba-common 
  samba-common-bin samba-libs smbclient sudo systemd systemd-sysv sysv-rc sysvinit sysvinit-utils t1utils task-catalan 
  task-catalan-desktop task-desktop task-gnome-desktop task-print-server tasksel tasksel-data tcl8.6 tzdata tzdata-java udev upower 
  usb-modeswitch-data util-linux util-linux-locales uuid-dev virtualbox virtualbox-dkms virtualbox-guest-additions-iso 
  virtualbox-guest-dkms virtualbox-guest-utils virtualbox-guest-x11 virtualbox-qt whois xbrlapi xdg-utils xserver-common xserver-xephyr 
  xserver-xorg-core xserver-xorg-video-geode xserver-xorg-video-vmware xterm xwayland youtube-dl 
The following partially installed packages will be configured:
The following packages are RECOMMENDED but will NOT be installed:
  gnupg-curl uuid-runtime 
514 packages upgraded, 0 newly installed, 0 to remove and 1 not upgraded.
Need to get 0 B/705 MB of archives. After unpacking 19.9 MB will be freed.
Do you want to continue? [Y/n/?] 
Reading changelogs... Done               
Extracting templates from packages: 100%
Preconfiguring packages ...
(Reading database ... 291930 files and directories currently installed.)
Preparing to unpack .../debconf_1.5.56_all.deb ...
  File "/usr/bin/py3clean", line 4
SyntaxError: Non-ASCII character '\xc2' in file /usr/bin/py3clean on line 4, but no encoding declared; see http://python.org/dev/peps/pep-0263/ for details
dpkg: warning: subprocess old pre-removal script returned error exit status 1
dpkg: trying script from the new package instead ...
  File "/usr/bin/py3clean", line 4
SyntaxError: Non-ASCII character '\xc2' in file /usr/bin/py3clean on line 4, but no encoding declared; see http://python.org/dev/peps/pep-0263/ for details
dpkg: error processing archive /var/cache/apt/archives/debconf_1.5.56_all.deb (--unpack):
 subprocess new pre-removal script returned error exit status 1
  File "/usr/bin/py3compile", line 4
SyntaxError: Non-ASCII character '\xc2' in file /usr/bin/py3compile on line 4, but no encoding declared; see http://python.org/dev/peps/pep-0263/ for details
dpkg: error while cleaning up:
 subprocess installed post-installation script returned error exit status 1
Errors were encountered while processing:
E: Sub-process /usr/bin/dpkg returned an error code (1)
Failed to perform requested operation on package.  Trying to recover:
dpkg: error processing package debconf (--configure):
 package is in a very bad inconsistent state; you should
 reinstall it before attempting configuration
Setting up gnome-menus (3.13.3-6) ...
Traceback (most recent call last):
  File "/usr/sbin/gnome-menus-blacklist", line 12, in <module>
    with open(blacklist_file, 'r', encoding='utf_8') as fp:
TypeError: 'encoding' is an invalid keyword argument for this function
dpkg: error processing package gnome-menus (--configure):
 subprocess installed post-installation script returned error exit status 1
dpkg: dependency problems prevent configuration of locales:
 locales depends on debconf (>= 0.5) | debconf-2.0; however:
  Package debconf is not configured yet.
  Package debconf-2.0 is not installed.
  Package debconf which provides debconf-2.0 is not configured yet.

dpkg: error processing package locales (--configure):
 dependency problems - leaving unconfigured
dpkg: error processing package gnome-orca (--configure):
 package is in a very bad inconsistent state; you should
 reinstall it before attempting configuration
Errors were encountered while processing:
  • Please post the complete transcript of a failed installation/upgrade/update attempt, please. In the question, not in comments. – Faheem Mitha Apr 24 '15 at 8:41
  • And perhaps apt works only with python2, so I'd try installing that if you don't have it yet. – lynxlynxlynx Apr 24 '15 at 9:26
  • Thanks @FaheemMitha Output of aptitude dist-upgrade added in the question – cram1010 Apr 24 '15 at 9:35
  • @lynxlynxlynx sudo python --version returns Python 2.7.8 – cram1010 Apr 24 '15 at 9:36
  • Ok, based on the info provided, python isn't your problem, so forget about it. For reference, see unix.stackexchange.com/q/121180/4671. This might be easier to handle in the chat room. Can you hop into chat.stackexchange.com/rooms/26/unix-and-linux please? We can talk there. And ping me if/when you go there. – Faheem Mitha Apr 24 '15 at 9:50

After some time spent talking to the poster in the U&L chat room, it because obvious this was a Python problem, specifically an encoding problem. The issue was that the poster's Python 3 was refusing to handle utf8 characters in scripts correctly. The specific errors in the question above were for a utf8 character, I think the copyright symbol.

I then found https://stackoverflow.com/q/25814675/350713

and used the test script mentioned there, namely

filename = 'utf-8·filename.txt'

On my machine this errored out with Python 2 and worked with Python 3, as follows.

python test.py 
  File "test.py", line 1
SyntaxError: Non-ASCII character '\xc2' in file test.py on line 1, but no encoding declared; see http://www.python.org/peps/pep-0263.html for details

python3 test.py 

However, the poster got the same error for both.

Additionally, the poster got ascii for both Python 2 and 3 when running

/usr/bin/python -c 'import sys; print(sys.getdefaultencoding())'

I got ascii for Python 2 and utf8 for Python 3.

Not knowing what was going on here, I then asked the experts in the Python room on Stack Overflow. Antti Haapala then asked some questions, and finally the following exchange occurred (http://chat.stackoverflow.com/transcript/message/22902726#22902726)

Antti Haapala   : @WaitingforDev... what is your python3 version?

WaitingforDev...: ooooops
                  I run python3 --version
                  and I get
                  Python 2.7.8

So the poster's Python 3 was actually Python 2, which explained the behavior. I then asked

FaheemMitha: what does `ls -la /usr/bin/python3` show?
Waiting for Dev...: @FaheemMitha it pointed to /usr/bin/python

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.