Unicode is a computing industry standard for the consistent encoding, representation and handling of text expressed in most of the world's writing systems.

learn more… | top users | synonyms (2)

0
votes
1answer
45 views

Can't use apt because python encoding issue

I'm getting a python unicode error every time I try to install or update anything with apt or aptitude, preventing me to update my system. The error I'm getting is the following: File ...
4
votes
1answer
33 views

Print out unicode values of stdin?

I use od to print me the octal or hex of a file/stdin/string. This lets me see the ASCII, or UTF-8 encoded, values of my stdin. But we don't live in ASCIIland anymore. Is there any command that will ...
4
votes
1answer
26 views

How to convert ęąśćżńł TXT to PDF with enscript?

I have problem converting a file to PDF. I create the file with echo ęąśćżńł > text and convert: enscript -O text -o - | ps2pdf - out.pdf However, out.pdf has an encoding problem:
13
votes
4answers
800 views

Is there an alternative to sed that supports unicode?

For example: sed 's/\u0091//g' file1 Right now, I have to do hexdump to get hex number and put into sed as follows: $ echo -ne '\u9991' | hexdump -C 00000000 e9 a6 91 ...
2
votes
0answers
75 views

xterm not displaying unicode

i have never been able to get my terminal to display unicode symbols. for example, before i had my present os, i mapped ctrl+a to the greek mu in vim, and it works on other computers, but not on my ...
1
vote
1answer
35 views

emacs text mode UTF-8 characters

I can't get emacs (version 24.4.1) in text mode (-nw launch option) to display Unicode characters directly. It outputs them in escaped form, e.g. ä becomes \344. When I save the file, it is encoded as ...
1
vote
0answers
40 views

How many bytes can a keypress return? And should a metafied key return more?

I've been tooling around w/ this input reader lately. Currently it only prints a report to stdout for each key pressed while it runs, but I would like to graduate it to a higher purpose sooner than ...
0
votes
1answer
40 views

*ls* font differences between files and folders with “--color” option

I have recently switched to urxvt from xterm for better unicode support and it does much better as to the font fallback. However I find an interesting issue: names of files and folders are rendered ...
1
vote
1answer
31 views

Any way to interpret pstree?

-bash-3.2$ pstree 29838 RBcron.shâââjavaâââ15*[{java}] -bash-3.2$ pstree 24423 bashââ¬âRBcron.shâââjavaâââ15*[{java}] ââvim -bash-3.2$ Any way to interpret this? Why would this happen?
3
votes
0answers
35 views

Emacs automatically adding BOM to XML file

This appears to be related to this discussion but I have exactly the opposite use case. I do not wish to insert a BOM but emacs appears to be inserting one in XML mode (without even being in ...
5
votes
2answers
410 views

Is there a universal way to write Unicode characters?

I sometimes want to write special Unicode characters, like an em-dash (—), in various programs like vim, Chrome (like just now!), or LibreOffice. In LibreOffice, I can achieve this via a dedicated ...
1
vote
2answers
245 views

How to convert unknown-8bit file to utf8

I have a .srt file that displays as gibberish when I open it in gEdit in ubuntu. So I want to convert it to utf8 to be able to read it. When I try to figure out what the encoding it give: file -i ...
1
vote
1answer
56 views

Detect how much of Unicode my terminal supports, even through screen

Here's the problem: I want to be able to discern if my terminal is capable of decent unicode or not, in order to use some characters or not, much like glances does, that sometimes uses colors and ...
1
vote
1answer
63 views

OpenBSD 5.6 / fvwm2 / xterm missing UTF-8 support

If I log on from a RHEL 6.5/GNOME notebook via SSH using gnome-terminal to an OpenBSD install, the UTF-8 support is OK. But if I use OpenBSD / fvwm2 / xterm locally, not via SSH, then it just misses ...
0
votes
1answer
74 views

Running Linux in text mode - blank diamonds instead of Chinese/Punjabi etc. characters

While in the text mode and using, for example, Chinese/Simplified or India/Punjabi locale I get blank diamond symbols in the console in place of the characters. I am running Xubuntu with ...
3
votes
1answer
62 views

How to search and replace dual characters by Unicode single characters in a garbled file?

I have a very long text file in french language that I need to clean up. The non ASCII characters have been replaced by combination of odd characters. As an example, the following content : passer de ...
2
votes
2answers
66 views

Free fonts with U+0CA0 (ಠ) character?

The glyph ಠ comes from the alphabet of Kannada, a language which I do not speak, and if it weren't for the Internet as it exists, I wouldn't care to have any of its glyphs. I recently discovered that ...
0
votes
0answers
68 views

how to install unicode fonts in gentoo-based linux?

this question: How can I get full Unicode font coverage for Gentoo Linux? gives the idea of what I want to do, but I don't know HOW to do that. I wish to install all unicode fonts, because github ...
3
votes
0answers
50 views

Is it safe to use tar even if there are some characters other than ASCII in filenames and paths?

Is it safe to use tar even if there are some characters other than ASCII printable characters? For example, Japanese characters, Chinese characters, newline character etc. Are there any known ...
1
vote
1answer
669 views

How to check if file has a BOM in utf-8 text

How can I check if a utf-8 text file have a BOM from command line? file command shows me UTF-8 Unicode text. But I don't know it means there is no BOM in the file. I'm using Ubuntu 12.04.
1
vote
1answer
93 views

Can't input non-latin characters into skype on OpenSUSE 13.1

I'm currently having an annoying issue with the latest version of Skype 4.3.0.37 (installed from the RPM on a 64bit machine following the official documentation). For some unknown to me reason, I'm ...
2
votes
0answers
262 views

How to set LANG in locale, despite it's in /etc/default/locale in ubuntu for unicode support?

I run Ubuntu 12.04, I can't set the locale to en_US.utf8. I set the LANG by configuring /etc/default/locale as described here. $cat /etc/default/locale LANG=en_US.utf8 $locale LANG= LANGUAGE= ...
5
votes
1answer
179 views

Can not use `cut -c` (`--characters`) with UTF-8?

The command cut has an option -c to work on characters, instead of bytes with the option -b. But that does not seem to work, in en_US.UTF-8 locale: The second byte gives the second ASCII character ...
3
votes
1answer
148 views

How to remove Russian and Arabic strings from a text file

Problem: I have text file with approximately 1M lines, each line consists of multiple words Some lines contain Russian or Arabic words (lets call them "bad lines"), I want to remove only these bad ...
3
votes
1answer
74 views

Find the best font for rendering a codepoint

How to find the appropriate font for rendering unicode codepoints ? gnome-terminal find that characters like «🉃⼼😻🕲🝤» can be rendered with fonts like Symbola rather than my terminal font or the ...
0
votes
2answers
245 views

Conversion of plaint-text files from ASCII to Unicode without any command

Why are ASCII-encoded files extended to UTF-8 or in reverse reduced to ASCII? user:~$ echo 'A B C | } ~' > ./file user:~$ user:~$ file --brief --mime ./file text/plain; charset=us-ascii ...
0
votes
0answers
45 views

Unicode emoji not showing up in tcsh prompt

I set my prompt on tcsh 6.18.01 to use some silly emoji characters, but they don't show up. > cat .cshrc set prompt = '\n🔥 [%~]\n%# ' \360\237\224\245 [~] ...
1
vote
2answers
81 views

Emacs shows octal escapes for some characters in a UTF-8 file

I copied some math formulas into some files under gedit, using the default settings of gedit from websites, such as math.stackexchange.com, wikipedia, ... When I open the files now under emacs, the ...
1
vote
2answers
177 views

Print UTF-8 file with command line tools

I want to print a file using a command like cat filename > /dev/lp0 where the file contains characters like ùàç encoded in UTF-8. Is this is possible or do I need to use CUPS? I'm using an ...
2
votes
2answers
77 views

Characters are encoded twice when I ask SSH to reattach a screen session on the remote host

I want to be able to SSH to a remote host and restore a screen session with one command. Both hosts use UTF-8 locale. My problem is that then, inside the screen session, characters are encoded twice. ...
1
vote
1answer
975 views

copy filenames with special characters to an external ntfs volume

I'm trying to copy the Documents and Settings folder of a Windows XP system over to an NTFS external disk using a USB Live of Puppy Linux. I encountered am encoding problem for namefiles in which the ...
11
votes
4answers
530 views

support for utf-8 encoding with lpr

While trying to send a text file to the printer via lpr from xterm, the content was corrupted beyond recognition, the cause of which was ultimately traced to the encoding of the file. If I instead ...
5
votes
1answer
2k views

How to display Chinese characters correctly on remote Red-Hat machine?

I am using Ubuntu14.04 to connect to a remote host. Which its version is: Linux version 2.6.32-431.11.5.el6.yyyzzz.x86_64 (gcc version 4.4.7 20120313 (Red Hat 4.4.7-4) (GCC) ) #1 SMP Thu Jul 3 ...
1
vote
1answer
53 views

What standard shell command can evaluate escaped Unicode in a stream?

I'm looking for a pipeable command that will convert substrings in the form of ‘\unnnn’ to the actual characters. For example: $ echo 'co\u00ebfficient' | what_command coëfficient
1
vote
2answers
5k views

iconv illegal input sequence- why?

While trying to convert a text file into its ASCII equivalent, I get error message that iconv: illegal input sequence at position. Command I use is iconv -f UTF-8 -t ascii//TRANSLIT file The ...
2
votes
1answer
48 views

How to display halfwidth Unicode characters in gnome-terminal etc

I'm wanting to set up vim with conceal to display the unicode characters U+2329 (〈) and U+232A (〈) in place of < and > when used to indicate type parameters. These are half-width characters that ...
8
votes
2answers
781 views

How can I make unicode symbols and truetype fonts work in xterm/uxterm?

I'd like to use a truetype font in xterm/uxterm (version 278) but I get problems with unicode symbols (é for example displays fine, so I guess it's not a disabled utf8 or a locale issue): $ echo -e ...
3
votes
1answer
1k views

Process a file that starts with a BOM (FF FE)

I received a .csv file with the FF FE BOM: $ head -n1 dotan.csv | hd 00000000 ff fe 41 00 64 00 20 00 67 00 72 00 6f 00 75 00 |..A.d. .g.r.o.u.| When using awk to parse it I'm getting a bunch of ...
2
votes
1answer
47 views

Unicode support in talk?

I live in Poland, and we have some accented characters - ąćęłńóśźż. Due to various reasons I resurrected "talk" usage on my server, which works great, but I wasn't able to figure out how to make it ...
6
votes
1answer
114 views

Heirloom Toolchest tr: error(s) trying to delete the complement of a set containing a multibyte character?

I'm trying to use the tr command from the Heirloom Toolchest to overcome a current limitation of the coreutils implementation, so as to be able to "pump" (with the -dc options) multibyte characters ...
0
votes
0answers
52 views

Spamassassin blacklist_subject not translating UTF8 characters

I am using spamassassin version 3.3.1 in conjunction with amavisd on Centos 6.5 64bit. It appears that when I use the whitelist subject plugin (actually using it to blacklist subjects, ie ...
1
vote
0answers
112 views

Is it possible to view unicode text in tcpdump capture output?

Unfortunately seems like tcpdump is not capable of detecting and displaying Unicode-encoded text (specifically utf-8) in captures. It treats unicode text like regular binary data and displays it in ...
2
votes
2answers
372 views

utf-8 sequence in the console

when I perform the "tree" command in the console, here's what I've got : . ├── Annexe\ 1\ -\ Sch\303\251ma\ global\ de\ la\ base\ de\ donn\303\251es.raw ... The result is composed of utf-8 ...
1
vote
0answers
48 views

Add Solaris utf-8 support for Postgresql

I need to use utf-8, but "locale -a" produces: C POSIX iso_8859_1 the os is: SunOS cteraportal 5.11 NexentaOS_134f i86pc i386 i86pc Solaris Additional info: root@cteraportal:~# dpkg -l locales ...
3
votes
2answers
230 views

cp: what quotation marks are used in error messages?

I'm currently fiddling around with a dying harddisk, and while trying to cp data from it, I get errors like cp: error reading ‘brokenFile’: Input/output error # comparison: backtick: `, apostrophe ' ...
4
votes
1answer
1k views

ImageMagick (or other) for UTF-8 text to image

I'm trying to create many single-character images using convert. The purpose is to provide a plastics specialist with images to print onto keyboard keycaps. The ImageMagick documentation suggests ...
1
vote
1answer
120 views

Recovering Mojibaked Files

I have recently been encountering several broken files whose contents got thoroughly baked in conversions between UTF-8 and ISO-Latin-1. How can I repair these files? The process that created the ...
2
votes
2answers
162 views

English Fonts with Persian Glyph

I'm trying to solve this problem for sometime. In the title bar of my web browsers and places like that; If there is a Farsi (Persian) string, it is shown like **** or something similar. I searched ...
3
votes
1answer
201 views

losing unicode support urxvt when logged in as user, root shell has them

I am on a minimal debian testing install. I've set up rxvt-unicode, herbstluftwm, and SLiM. With my username, after SLiM logs me in, all of the urxvt terminal instances i start lack unicode support. ...
1
vote
1answer
49 views

Why doesn't `restore` display my UTF-8 encoded filenames correctly in interactive mode?

When using the interactive mode of the restore utility to restore backups made with dump UTF-8-encoded filenames doesn't display correctly, see example below. The ??s should be ös ... $ sudo restore ...