Unicode is a computing industry standard for the consistent encoding, representation and handling of text expressed in most of the world's writing systems.

learn more… | top users | synonyms (2)

0
votes
2answers
41 views

iconv illegal input sequence- why?

While trying to convert a text file into its ASCII equivalent, I get error message that iconv: illegal input sequence at position. Command I use is iconv -f UTF-8 -t ascii//TRANSLIT file The ...
2
votes
0answers
17 views

How to display halfwidth Unicode characters in gnome-terminal etc

I'm wanting to set up vim with conceal to display the unicode characters U+2329 (〈) and U+232A (〈) in place of < and > when used to indicate type parameters. These are half-width characters that ...
8
votes
2answers
129 views

How can I make unicode symbols and truetype fonts work in xterm/uxterm?

I'd like to use a truetype font in xterm/uxterm (version 278) but I get problems with unicode symbols (é for example displays fine, so I guess it's not a disabled utf8 or a locale issue): $ echo -e ...
3
votes
1answer
167 views

Process a file that starts with a BOM (FF FE)

I received a .csv file with the FF FE BOM: $ head -n1 dotan.csv | hd 00000000 ff fe 41 00 64 00 20 00 67 00 72 00 6f 00 75 00 |..A.d. .g.r.o.u.| When using awk to parse it I'm getting a bunch of ...
2
votes
1answer
23 views

Unicode support in talk?

I live in Poland, and we have some accented characters - ąćęłńóśźż. Due to various reasons I resurrected "talk" usage on my server, which works great, but I wasn't able to figure out how to make it ...
6
votes
1answer
67 views

Heirloom Toolchest tr: error(s) trying to delete the complement of a set containing a multibyte character?

I'm trying to use the tr command from the Heirloom Toolchest to overcome a current limitation of the coreutils implementation, so as to be able to "pump" (with the -dc options) multibyte characters ...
0
votes
0answers
20 views

Spamassassin blacklist_subject not translating UTF8 characters

I am using spamassassin version 3.3.1 in conjunction with amavisd on Centos 6.5 64bit. It appears that when I use the whitelist subject plugin (actually using it to blacklist subjects, ie ...
1
vote
0answers
28 views

Is it possible to view unicode text in tcpdump capture output?

Unfortunately seems like tcpdump is not capable of detecting and displaying Unicode-encoded text (specifically utf-8) in captures. It treats unicode text like regular binary data and displays it in ...
2
votes
2answers
177 views

utf-8 sequence in the console

when I perform the "tree" command in the console, here's what I've got : . ├── Annexe\ 1\ -\ Sch\303\251ma\ global\ de\ la\ base\ de\ donn\303\251es.raw ... The result is composed of utf-8 ...
1
vote
0answers
13 views

utf-8 locale missing on nexenta

I need to use utf-8, but "locale -a" produces: C POSIX iso_8859_1 the os is: SunOS cteraportal 5.11 NexentaOS_134f i86pc i386 i86pc Solaris Additional info: root@cteraportal:~# dpkg -l locales ...
3
votes
2answers
183 views

cp: what quotation marks are used in error messages?

I'm currently fiddling around with a dying harddisk, and while trying to cp data from it, I get errors like cp: error reading ‘brokenFile’: Input/output error # comparison: backtick: `, apostrophe ' ...
4
votes
1answer
175 views

ImageMagick (or other) for UTF-8 text to image

I'm trying to create many single-character images using convert. The purpose is to provide a plastics specialist with images to print onto keyboard keycaps. The ImageMagick documentation suggests ...
1
vote
1answer
28 views

Recovering Mojibaked Files

I have recently been encountering several broken files whose contents got thoroughly baked in conversions between UTF-8 and ISO-Latin-1. How can I repair these files? The process that created the ...
2
votes
2answers
80 views

English Fonts with Persian Glyph

I'm trying to solve this problem for sometime. In the title bar of my web browsers and places like that; If there is a Farsi (Persian) string, it is shown like **** or something similar. I searched ...
2
votes
1answer
62 views

losing unicode support urxvt when logged in as user, root shell has them

I am on a minimal debian testing install. I've set up rxvt-unicode, herbstluftwm, and SLiM. With my username, after SLiM logs me in, all of the urxvt terminal instances i start lack unicode support. ...
1
vote
1answer
35 views

Why doesn't `restore` display my UTF-8 encoded filenames correctly in interactive mode?

When using the interactive mode of the restore utility to restore backups made with dump UTF-8-encoded filenames doesn't display correctly, see example below. The ??s should be ös ... $ sudo restore ...
3
votes
2answers
297 views

How to make the login shell xterm use utf-8?

I use xterm (X-Win32 2012 Build 30 from StarNet Communications Corp) to login from a Windows 7 PC to a Red Enterprise Linux 6 (RHEL6). My problem is that all multi-byte utf-8 characters comes out ...
1
vote
2answers
115 views

how to put color variables in bash $'…'

I have defined in one file which I source from .bashrc, color constants.. like these.. txtgrn='\e[0;32m' # Green txtylw='\e[0;33m' # Yellow txtblu='\e[0;34m' # Blue txtpur='\e[0;35m' # ...
1
vote
1answer
131 views

UTF-8 in Terminator

I use the Terminator terminal emulator, and out of pure boredom, I installed and attempted to run this program: https://github.com/thiderman/doge However, it requires UTF-8 encoding to work. I opened ...
2
votes
2answers
157 views

Missing unicode characters

I've just switched from KDE to lightweight slim+xmonad environment and my fonts are not rendering correctly. I use DejaVu font family (default for Debian 7.x) which supports all characters I need ...
2
votes
1answer
245 views

How can I make the TTY use the appropriate charset?

Note: forked of this per recommendation here. While I got the X keyboard layout right on my own after I fixed that stupid typo, I still can't set the console charset, key map, font map, or whatever ...
0
votes
1answer
816 views

Problems with the locale; can't make Arch Linux use UFT-8 [closed]

Note: The real problems behind this are presented here, as per a recommendation on meta. This question should be closed as a non-reproducible off-topic because its main problem is solved by correcting ...
5
votes
1answer
402 views

Non-printable-ASCII characters in sshd banner

It's possible to configure a banner for sshd that is to be displayed as a connection is opened, via Banner /etc/motd.ssh in sshd_config. Note that this is displayed before the authentication occurs, ...
1
vote
1answer
264 views

Can vim display ASCII characters only, and treat other bytes as binary data?

I already know vim -b, however, depending on the locale used, it displays multi-byte characters (like UTF-8) as single letters. How can I ask vim to only display ASCII printable characters, and ...
3
votes
1answer
185 views

gnome-terminal unicode chars not showing - Locale is set to en_US.utf8

I've become stumped trying to get my terminal to show unicode characters. I've set the LOCALE/LANG environment variables to en_US.utf8: locale LANG=en_US.utf8 LC_CTYPE=en_US.utf8 ...
2
votes
1answer
49 views

Can't obtain the connected forms of some unicode grapheme clusters with combining diacritical marks

In Arabic and some other languages a character has two or three forms which will find its appropriate one automatically when you type a word (according to its previous and next characters). How can ...
4
votes
2answers
757 views

How to mount an NTFS drive as utf8 over NFS

Question How to mount a nfs share (NTFS formatted fileystem) as utf8 in CentOS 6.5? What I tried mount -o iocharset=utf8 mount -o nfs=utf8 mount -o utf8=1 mount -o utf8 Problem: When I ...
3
votes
4answers
272 views

Do any text-based web browsers support unicode box-drawing characters?

I have some html like this: &#x250c&#x2500&#x2500&#x2500&#x2500&#x2500&#x2500&#x2500&#x2500 ...
3
votes
1answer
148 views

Printing UTF-8 (including Chinese) text

I am trying to print a large quantity (several megabytes) of UTF-8 encoded text which consists of Chinese and Latin characters (and maybe a sprinkling of others). I would like to print it in several ...
3
votes
1answer
108 views

Are there terminal emulators that support bi-directional text?

I tried searching for this but came out empty-handed. Is it possible for me to get my terminal emulator to display right-to-left Unicode grapheme clusters properly? Use case: I'm writing a script ...
2
votes
2answers
153 views

Is it possible to show mathematical symbols in the terminal?

I want to do some program that can help me to check scientific formulars via terminal. It is possible or not if i want to show mathematical symbols (e.g. integral, diff Greek's Alphabets) on Linux's ...
5
votes
2answers
387 views

Convert between Unicode Normalization Forms on the unix command-line

In Unicode, some character combinations have more than one representation. For example, the character ä can be represented as "ä", that is the codepoint U+00E4 (two bytes c3 a4 in UTF-8 encoding), ...
2
votes
1answer
482 views

Double line box_drawing characters in terminal

This piece of code in wikipedia: $ char=( 6a 6b 6c 6d 6e 71 74 75 76 77 78 ) $ for i in ${char[*]}; do printf "0x$i \x$i \e(0\x$i\e(B\n"; done includes one-line box-drawing characters. Are there ...
4
votes
2answers
200 views

Swedish unicode characters in xdm / xlogin

I decided to localize my XDM login screen but I'm stuck with corrupted Swedish characters. I'm using Debian Wheezy with the following /etc/X11/xdm/Xresources Xcursor.theme: whiteglass ...
2
votes
1answer
104 views

console displaying quote character from gcc wongly

After an upgrade to debian wheezy my text consoles displays questions marks where there should be quotes Under X I see things correctly window.cpp:1056:21: error: base operand of ‘->’ is not a ...
2
votes
2answers
637 views

Some UTF-8 characters not being recognized by grep or sed

Trying to determine all characters in a file. The file sample consists of: a eɪ abandon əˈbændən ability əˈbɪləti able ˈeɪbəl able ˈeɪbl abortion əˈbɔrʃən abortion əˈbɔrʃn about əˈbaʊt above əˈbʌv ...
1
vote
2answers
5k views

Converting from ascii to utf-8 format - iconv not working [closed]

I have a requirement to convert from ASCII text format to UTF-8. Below is what I am performing through the iconv command: [root@main tmp]# cat File1 1 5 6 [root@main tmp]# file File1 File1: ASCII ...
1
vote
1answer
241 views

Non-ASCII characters handled incorrectly on command line

I have a problem with non-ASCII characters in the Terminal on Mac. If I write: echo ÆØÅ Then press Enter, Arrow Up, Arrow Left, so the cursor is between the Ø and the Å, then write X, then the Å is ...
5
votes
2answers
622 views

How to convert UTF-8 txt files to all uppercase in bash?

I have some UTF-8 .txt files which I would like to convert to all uppercase. If it was just ASCII, I could use: tr [:lower:] [:upper:] But since I'm working with diacritics and stuff, it doesn't ...
3
votes
1answer
78 views

Squid doesn't block non-ascii domans

I have the following lines in /etc/squid/squid.conf: acl bad_url url_regex "/etc/squid/block.acl" http_access deny bad_url All of the urls in block.acl must be blocked by the proxy server. It works ...
2
votes
1answer
407 views

GNU Screen doesn't echo unicode characters correct

When I do this in my terminal: echo -e "\xF0\x9f\x8d\xba" I get a cool beer mug in my terminal: 🍺 However, when I start screen, and try the same, I'm getting a �. How do I get the nice beer mug? ...
2
votes
1answer
189 views

Incorrect sorting according en_US.UTF-8

I'm seeing a weird behavior of the sort command when I sort by en_US.UTF-8. I'm trying to sort this file (LC_COLLATE=en_US.UTF-8 sort testdata): 㐀 㐃㐄 ▼▽◢ ㎏㎜㎝ bqz ㄠㄨㄩ─┃┆ ...
5
votes
3answers
257 views

How can I get rid of this invisible file?

Background I have a folder named akorg✽. That Unicode character causes headaches for me when software makes incorrect assumptions about the text encoding of my file paths, so I'd like to remove it ...
3
votes
1answer
290 views

UTF-8 Characters in Byobu Screen Status Line

I'm running Byobu with Screen backend. I have a custom status script that wget's an RSS feed and shows the latest headline in it. If there are non-ASCII characters, they are not printed correctly. ...
5
votes
2answers
103 views

Keyboard combination to insert non keyboard characters

on windows one could with Alt+235 print ë into (any) textfield. I tried the same on linux and it seems to be not working. Depending on the application the result is different, but never the wanted ...
0
votes
2answers
920 views

what is “an invisible whitespace character that takes up space”

One can look up Unicode Characters with Regular Expressions. On Jan Goyvaerts website I found a RegExp whose meaning I don't understand : \p{Zs} or \p{Space_Separator}: a whitespace character that ...
2
votes
2answers
281 views

Sort lines by unicode value

I'm trying to sort a text file linewise by their Unicode values. As far as I can tell, this means numerals first, then letters, then CJK-Ideographs. However, sort (with LC_ALL=C) fails horribly at ...
4
votes
2answers
428 views

What would break if the C locale was UTF-8 instead of ASCII?

The C locale is defined to use the ASCII charset and POSIX does not provide a way to use a charset without changing the locale as well. What would happen if the encoding of C were switched to UTF-8 ...
2
votes
3answers
384 views

Insert Unicode Characters at Console Login

My password contains "special" characters like ü. How can I login at the console (no X running) when the default keyboard layout does not provide these characters? Or, differently put: how can I ...
11
votes
2answers
1k views

grep: Find all lines that contain Japanese kanjis

In a huge UTF-8 text file, I want to show all lines that contains Japanese kanjis. What grep (or other) expression does this? If I am not mistaken, kanjis are the characters between \u4e00 and ...