Unicode is a computing industry standard for the consistent encoding, representation and handling of text expressed in most of the world's writing systems.

learn more… | top users | synonyms (2)

1
vote
0answers
13 views

SQLplus and oracle,bad utf8 display

I have set export NLS_LANG=Italian_ITALY.UTF8 db is on utf8 SELECT * FROM NLS_DATABASE_PARAMETERS NLS_CHARACTERSET |AL32UTF8 But ...
1
vote
1answer
30 views

Cannot get vim to recognize unicode

I am having the most trouble ever getting vim to recognize unicode. At the top of my vimrc I have set encoding=utf-8 set fileencoding=utf-8 and LANG shows echo $LANG en_US.UTF-8 but when I open ...
2
votes
1answer
19 views

Cyrillic file names broken after upgrading Slackware

After upgrading glibc and some other libraries on Slackware instead of cyrillic names I see the garbage: | Was before | Is now | |------------+------------------| | Вождение | ...
0
votes
1answer
104 views

Can't use apt because python encoding issue

I'm getting a python unicode error every time I try to install or update anything with apt or aptitude, preventing me to update my system. The error I'm getting is the following: File ...
4
votes
1answer
39 views

Print out unicode values of stdin?

I use od to print me the octal or hex of a file/stdin/string. This lets me see the ASCII, or UTF-8 encoded, values of my stdin. But we don't live in ASCIIland anymore. Is there any command that will ...
4
votes
1answer
32 views

How to convert ęąśćżńł TXT to PDF with enscript?

I have problem converting a file to PDF. I create the file with echo ęąśćżńł > text and convert: enscript -O text -o - | ps2pdf - out.pdf However, out.pdf has an encoding problem:
13
votes
4answers
858 views

Is there an alternative to sed that supports unicode?

For example: sed 's/\u0091//g' file1 Right now, I have to do hexdump to get hex number and put into sed as follows: $ echo -ne '\u9991' | hexdump -C 00000000 e9 a6 91 ...
2
votes
0answers
100 views

xterm not displaying unicode

i have never been able to get my terminal to display unicode symbols. for example, before i had my present os, i mapped ctrl+a to the greek mu in vim, and it works on other computers, but not on my ...
1
vote
1answer
42 views

emacs text mode UTF-8 characters

I can't get emacs (version 24.4.1) in text mode (-nw launch option) to display Unicode characters directly. It outputs them in escaped form, e.g. ä becomes \344. When I save the file, it is encoded as ...
1
vote
0answers
45 views

How many bytes can a keypress return? And should a metafied key return more?

I've been tooling around w/ this input reader lately. Currently it only prints a report to stdout for each key pressed while it runs, but I would like to graduate it to a higher purpose sooner than ...
0
votes
1answer
47 views

*ls* font differences between files and folders with “--color” option

I have recently switched to urxvt from xterm for better unicode support and it does much better as to the font fallback. However I find an interesting issue: names of files and folders are rendered ...
1
vote
1answer
39 views

Any way to interpret pstree?

-bash-3.2$ pstree 29838 RBcron.shâââjavaâââ15*[{java}] -bash-3.2$ pstree 24423 bashââ¬âRBcron.shâââjavaâââ15*[{java}] ââvim -bash-3.2$ Any way to interpret this? Why would this happen?
3
votes
0answers
47 views

Emacs automatically adding BOM to XML file

This appears to be related to this discussion but I have exactly the opposite use case. I do not wish to insert a BOM but emacs appears to be inserting one in XML mode (without even being in ...
5
votes
2answers
432 views

Is there a universal way to write Unicode characters?

I sometimes want to write special Unicode characters, like an em-dash (—), in various programs like vim, Chrome (like just now!), or LibreOffice. In LibreOffice, I can achieve this via a dedicated ...
1
vote
2answers
373 views

How to convert unknown-8bit file to utf8

I have a .srt file that displays as gibberish when I open it in gEdit in ubuntu. So I want to convert it to utf8 to be able to read it. When I try to figure out what the encoding it give: file -i ...
1
vote
1answer
72 views

Detect how much of Unicode my terminal supports, even through screen

Here's the problem: I want to be able to discern if my terminal is capable of decent unicode or not, in order to use some characters or not, much like glances does, that sometimes uses colors and ...
1
vote
1answer
87 views

OpenBSD 5.6 / fvwm2 / xterm missing UTF-8 support

If I log on from a RHEL 6.5/GNOME notebook via SSH using gnome-terminal to an OpenBSD install, the UTF-8 support is OK. But if I use OpenBSD / fvwm2 / xterm locally, not via SSH, then it just misses ...
0
votes
1answer
83 views

Running Linux in text mode - blank diamonds instead of Chinese/Punjabi etc. characters

While in the text mode and using, for example, Chinese/Simplified or India/Punjabi locale I get blank diamond symbols in the console in place of the characters. I am running Xubuntu with ...
3
votes
1answer
67 views

How to search and replace dual characters by Unicode single characters in a garbled file?

I have a very long text file in french language that I need to clean up. The non ASCII characters have been replaced by combination of odd characters. As an example, the following content : passer de ...
2
votes
2answers
76 views

Free fonts with U+0CA0 (ಠ) character?

The glyph ಠ comes from the alphabet of Kannada, a language which I do not speak, and if it weren't for the Internet as it exists, I wouldn't care to have any of its glyphs. I recently discovered that ...
0
votes
0answers
79 views

how to install unicode fonts in gentoo-based linux?

this question: How can I get full Unicode font coverage for Gentoo Linux? gives the idea of what I want to do, but I don't know HOW to do that. I wish to install all unicode fonts, because github ...
3
votes
1answer
55 views

Is it safe to use tar even if there are some characters other than ASCII in filenames and paths?

Is it safe to use tar even if there are some characters other than ASCII printable characters? For example, Japanese characters, Chinese characters, newline character etc. Are there any known ...
1
vote
1answer
887 views

How to check if file has a BOM in utf-8 text

How can I check if a utf-8 text file have a BOM from command line? file command shows me UTF-8 Unicode text. But I don't know it means there is no BOM in the file. I'm using Ubuntu 12.04.
1
vote
1answer
113 views

Can't input non-latin characters into skype on OpenSUSE 13.1

I'm currently having an annoying issue with the latest version of Skype 4.3.0.37 (installed from the RPM on a 64bit machine following the official documentation). For some unknown to me reason, I'm ...
2
votes
0answers
310 views

How to set LANG in locale, despite it's in /etc/default/locale in ubuntu for unicode support?

I run Ubuntu 12.04, I can't set the locale to en_US.utf8. I set the LANG by configuring /etc/default/locale as described here. $cat /etc/default/locale LANG=en_US.utf8 $locale LANG= LANGUAGE= ...
5
votes
1answer
213 views

Can not use `cut -c` (`--characters`) with UTF-8?

The command cut has an option -c to work on characters, instead of bytes with the option -b. But that does not seem to work, in en_US.UTF-8 locale: The second byte gives the second ASCII character ...
3
votes
1answer
163 views

How to remove Russian and Arabic strings from a text file

Problem: I have text file with approximately 1M lines, each line consists of multiple words Some lines contain Russian or Arabic words (lets call them "bad lines"), I want to remove only these bad ...
3
votes
1answer
79 views

Find the best font for rendering a codepoint

How to find the appropriate font for rendering unicode codepoints ? gnome-terminal find that characters like «🉃⼼😻🕲🝤» can be rendered with fonts like Symbola rather than my terminal font or the ...
0
votes
2answers
274 views

Conversion of plaint-text files from ASCII to Unicode without any command

Why are ASCII-encoded files extended to UTF-8 or in reverse reduced to ASCII? user:~$ echo 'A B C | } ~' > ./file user:~$ user:~$ file --brief --mime ./file text/plain; charset=us-ascii ...
0
votes
0answers
52 views

Unicode emoji not showing up in tcsh prompt

I set my prompt on tcsh 6.18.01 to use some silly emoji characters, but they don't show up. > cat .cshrc set prompt = '\n🔥 [%~]\n%# ' \360\237\224\245 [~] ...
1
vote
2answers
82 views

Emacs shows octal escapes for some characters in a UTF-8 file

I copied some math formulas into some files under gedit, using the default settings of gedit from websites, such as math.stackexchange.com, wikipedia, ... When I open the files now under emacs, the ...
1
vote
2answers
193 views

Print UTF-8 file with command line tools

I want to print a file using a command like cat filename > /dev/lp0 where the file contains characters like ùàç encoded in UTF-8. Is this is possible or do I need to use CUPS? I'm using an ...
2
votes
2answers
85 views

Characters are encoded twice when I ask SSH to reattach a screen session on the remote host

I want to be able to SSH to a remote host and restore a screen session with one command. Both hosts use UTF-8 locale. My problem is that then, inside the screen session, characters are encoded twice. ...
1
vote
1answer
1k views

copy filenames with special characters to an external ntfs volume

I'm trying to copy the Documents and Settings folder of a Windows XP system over to an NTFS external disk using a USB Live of Puppy Linux. I encountered am encoding problem for namefiles in which the ...
11
votes
4answers
599 views

support for utf-8 encoding with lpr

While trying to send a text file to the printer via lpr from xterm, the content was corrupted beyond recognition, the cause of which was ultimately traced to the encoding of the file. If I instead ...
5
votes
1answer
2k views

How to display Chinese characters correctly on remote Red-Hat machine?

I am using Ubuntu14.04 to connect to a remote host. Which its version is: Linux version 2.6.32-431.11.5.el6.yyyzzz.x86_64 (gcc version 4.4.7 20120313 (Red Hat 4.4.7-4) (GCC) ) #1 SMP Thu Jul 3 ...
1
vote
1answer
55 views

What standard shell command can evaluate escaped Unicode in a stream?

I'm looking for a pipeable command that will convert substrings in the form of ‘\unnnn’ to the actual characters. For example: $ echo 'co\u00ebfficient' | what_command coëfficient
1
vote
2answers
5k views

iconv illegal input sequence- why?

While trying to convert a text file into its ASCII equivalent, I get error message that iconv: illegal input sequence at position. Command I use is iconv -f UTF-8 -t ascii//TRANSLIT file The ...
2
votes
1answer
48 views

How to display halfwidth Unicode characters in gnome-terminal etc

I'm wanting to set up vim with conceal to display the unicode characters U+2329 (〈) and U+232A (〈) in place of < and > when used to indicate type parameters. These are half-width characters that ...
8
votes
2answers
854 views

How can I make unicode symbols and truetype fonts work in xterm/uxterm?

I'd like to use a truetype font in xterm/uxterm (version 278) but I get problems with unicode symbols (é for example displays fine, so I guess it's not a disabled utf8 or a locale issue): $ echo -e ...
3
votes
1answer
1k views

Process a file that starts with a BOM (FF FE)

I received a .csv file with the FF FE BOM: $ head -n1 dotan.csv | hd 00000000 ff fe 41 00 64 00 20 00 67 00 72 00 6f 00 75 00 |..A.d. .g.r.o.u.| When using awk to parse it I'm getting a bunch of ...
2
votes
1answer
47 views

Unicode support in talk?

I live in Poland, and we have some accented characters - ąćęłńóśźż. Due to various reasons I resurrected "talk" usage on my server, which works great, but I wasn't able to figure out how to make it ...
6
votes
1answer
115 views

Heirloom Toolchest tr: error(s) trying to delete the complement of a set containing a multibyte character?

I'm trying to use the tr command from the Heirloom Toolchest to overcome a current limitation of the coreutils implementation, so as to be able to "pump" (with the -dc options) multibyte characters ...
0
votes
0answers
54 views

Spamassassin blacklist_subject not translating UTF8 characters

I am using spamassassin version 3.3.1 in conjunction with amavisd on Centos 6.5 64bit. It appears that when I use the whitelist subject plugin (actually using it to blacklist subjects, ie ...
1
vote
0answers
121 views

Is it possible to view unicode text in tcpdump capture output?

Unfortunately seems like tcpdump is not capable of detecting and displaying Unicode-encoded text (specifically utf-8) in captures. It treats unicode text like regular binary data and displays it in ...
2
votes
2answers
400 views

utf-8 sequence in the console

when I perform the "tree" command in the console, here's what I've got : . ├── Annexe\ 1\ -\ Sch\303\251ma\ global\ de\ la\ base\ de\ donn\303\251es.raw ... The result is composed of utf-8 ...
1
vote
0answers
49 views

Add Solaris utf-8 support for Postgresql

I need to use utf-8, but "locale -a" produces: C POSIX iso_8859_1 the os is: SunOS cteraportal 5.11 NexentaOS_134f i86pc i386 i86pc Solaris Additional info: root@cteraportal:~# dpkg -l locales ...
3
votes
2answers
235 views

cp: what quotation marks are used in error messages?

I'm currently fiddling around with a dying harddisk, and while trying to cp data from it, I get errors like cp: error reading ‘brokenFile’: Input/output error # comparison: backtick: `, apostrophe ' ...
4
votes
1answer
1k views

ImageMagick (or other) for UTF-8 text to image

I'm trying to create many single-character images using convert. The purpose is to provide a plastics specialist with images to print onto keyboard keycaps. The ImageMagick documentation suggests ...
1
vote
1answer
126 views

Recovering Mojibaked Files

I have recently been encountering several broken files whose contents got thoroughly baked in conversions between UTF-8 and ISO-Latin-1. How can I repair these files? The process that created the ...