emacs text mode UTF-8 characters

Question

I can't get emacs (version 24.4.1) in text mode (-nw launch option) to display Unicode characters directly. It outputs them in escaped form, e.g. ä becomes \344. When I save the file, it is encoded as ISO-8859, even though my system locale is en_US.UTF-8.

The terminal itself can display Unicode characters outside of emacs; e.g., in bash, ä becomes ä. It also works in the emacs X application (without -nw).

I'm running xterm -u8 (same result using uxterm) on Arch Linux 3.19.3-3 with gnome-shell 3.16.0-2.

localectl output:

   System Locale: LANG=en_US.UTF-8
       VC Keymap: sv-latin1
      X11 Layout: se
       X11 Model: pc104

locale output:

locale: Cannot set LC_CTYPE to default locale: No such file or directory
locale: Cannot set LC_MESSAGES to default locale: No such file or directory
locale: Cannot set LC_ALL to default locale: No such file or directory
LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=

Since the only problem area is emacs in terminal mode, I'm guessing that's where the problem is. Any ideas?

Update:

~/.emacs:

(prefer-coding-system       'utf-8)
(set-default-coding-systems 'utf-8)
(set-terminal-coding-system 'utf-8)
(set-keyboard-coding-system 'utf-8)

(setq x-select-request-type '(UTF8_STRING COMPOUND_TEXT TEXT STRING))
(custom-set-variables
 ;; custom-set-variables was added by Custom.
 ;; If you edit it by hand, you could mess it up, so be careful.
 ;; Your init file should contain only one such instance.
 ;; If there is more than one, they won't work right.
 '(ansi-color-names-vector
   ["#2d3743" "#ff4242" "#74af68" "#dbdb95" "#34cae2" "#008b8b" "#00ede1" "#e1e1e0"])
 '(current-language-environment "UTF-8")
 '(custom-enabled-themes (quote (manoj-dark)))
 '(font-use-system-font t)
 '(inhibit-startup-screen t)
 '(tool-bar-mode nil))
(custom-set-faces
 ;; custom-set-faces was added by Custom.
 ;; If you edit it by hand, you could mess it up, so be careful.
 ;; Your init file should contain only one such instance.
 ;; If there is more than one, they won't work right.
 )

I tried commenting out every line in the file, but the result was the same afterwards.

Answer 1

After some fiddling, I found it was system-related rather than emacs-related. Apparently my locale settings were messed up somehow, as we saw in the OP:

locale: Cannot set LC_CTYPE to default locale: No such file or directory
locale: Cannot set LC_MESSAGES to default locale: No such file or directory
locale: Cannot set LC_ALL to default locale: No such file or directory

What I did to resolve it:

In /etc/locale.gen, uncomment/add the line that says en_US.UTF-8 UTF-8. Make sure it’s the only uncommented line. It should work with any other UTF-8 setting if you’d rather use one of those. A brute force method would be:

$ sudo bash -c 'echo en_US.UTF-8 UTF-8 > /etc/locale.gen'

After this we generate the locale settings and store them in the system configuration:

$ sudo locale-gen
$ sudo bash -c 'locale > /etc/locale.conf'

You may be able to update the settings without rebooting, but I didn’t find a way.