2

Sometimes when I use the rm tool or stat tool in bash shell I get following weird characters:

root@server:~# rm -vr blah
removed directory: �blah�
root@server:~# 

Locale settings can be seen here:

root@server:~# locale -a
C
C.UTF-8
en_US.utf8
POSIX
root@server:~# locale
LANG=en_US.UTF-8
LANGUAGE=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=en_US.UTF-8
root@server:~# 

What might be the reason for this?

  • 2
    This is usually a wrong charter encoding. It can be difficult to set correctly, unless you know excatly what your doing, but your distro should have the ability to change language somewere, doing that (even if just changing it and changing it back) usually fixed character encoding. – coteyr Aug 27 '15 at 15:11
  • benjamin-schweizer.de/… – coteyr Aug 27 '15 at 15:11
  • Could you add the output of locale – Esref Aug 27 '15 at 17:14
  • @Esref I added output of locale. – Martin Aug 28 '15 at 7:23
  • 2
    @Esref, no, they're more probably (U+2018 LEFT SINGLE QUOTATION MARK) and (U+2019 RIGHT SINGLE QUOTATION MARK). Since only one replacement character is being displayed, the terminal emulator is probably correctly configured in UTF-8 but the font it's using is lacking those characters. What's the terminal emulator? – Stéphane Chazelas Aug 28 '15 at 8:40
1

Those characters are most likely (U+2018 LEFT SINGLE QUOTATION MARK) and (U+2019 RIGHT SINGLE QUOTATION MARK).

Since only one replacement character is being displayed, the terminal emulator is probably correctly configured in UTF-8 but the font it's using is lacking those characters. So the terminal uses the glyph for the replacement character instead.

With this helper:

describe_non_ascii() {
  perl -Mcharnames=full -Mopen=locale -lpe '
   s{
      [^[:ascii:]]
   }{
      sprintf("{U+%04X: %s}", ord($&),  charnames::viacode(ord($&)))
   }gex'
}

I can see:

$ rm -vr blah | describe_non_ascii
removed directory: {U+2018: LEFT SINGLE QUOTATION MARK}blah{U+2019: RIGHT SINGLE QUOTATION MARK}

In UTF-8, those characters are encoded on 3 bytes each:

$ printf '\u2018\u2019' | hd
00000000  e2 80 98 e2 80 99                                 |......|
00000006

So if the terminal was not correctly configured in UTF-8, I'd expect to see 3 glyphs, possibly three � if none of those bytes were characters in the charset the terminal expects for each.

| improve this answer | |
1

It could be the encoding on your terminal.
Example I use putty terminal and had to change to utf-8.

| improve this answer | |
  • I had the very same thing happen to me. The issue I had was my terminal, putty, character set was not correct. To fix it I changed it to utf-8. I checked my server (ubuntu) with 'locale' and got "LANG=en_US.UTF-8). – jc__ Aug 27 '15 at 17:04
  • This is right on point. I use different OS & had to make do on what I have access at the moment. This issue showed when using putty on windows. I know it's a character encoding issue and has been bugging me for a while but didn't put much time to figuring it out until now. The server is set on "LANG=en_US.UTF-8" but my putty client was using "ISO-8859-1:1998 (Latin-1, West Europe)" character set. No idea how that was set there but ching it back to "UTF-8" and making sure that this is the default setting on all new connections fixes the problem. share.getcloudapp.com/X6uObWge – kramfs May 4 at 5:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.