Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [unicode]

Unicode is a computing industry standard for the consistent encoding, representation and handling of text expressed in most of the world's writing systems.

3
votes
1answer
39 views

Neomutt - Can't display Swedish characters

I have an issue regarding neomutt not displaying the Swedish characters å, ä and ö. When typing locale in the terminal I get: LANG=sv_SE.UTF-8 LC_CTYPE=sv_SE.UTF-8 LC_NUMERIC=sv_SE.UTF-8 LC_TIME=...
-2
votes
0answers
40 views

Emoji on Debian

I have installed Debian 9 recently. The problem that I have is that some emoji are missing from the system. On Google Chrome: - I only have some emoji, this is a screen capture from https://eosrei....
0
votes
0answers
9 views

Is it possible to make bash support UTF-8 without any locale files installed?

I have installed the bash binary into an Android device (with sh as the filename), using an app that I don't fully trust. It interprets non-ASCII Unicode characters as random control codes, or has no ...
2
votes
1answer
56 views

Grep search for words containing capital letters in Unicode

After some searching, I found a grep command that will find words containing capital letters, and list them without duplicates: grep -hoP "\w*[[:upper:]]+\w*" * | sort -u However, when it comes to ...
0
votes
1answer
46 views

Distinguish ascii from UTF-8 characters in the same file

On Ubuntu 18.04, I created a dummy text file with just one UTF-8 character, è. The other characters are all ascii: $ cat dummytext Hello Helloè This is the resulting hexdump: $ hexdump -C dummyfile ...
1
vote
1answer
55 views

Recursively converting Windows files to Unix files

I have a PHP application with is located on Linux with multiple directories (and sub-directories) and many PHP, JS, HTML, CSS, etc files. Many of the files have Windows EOL control characters and I ...
0
votes
1answer
28 views

Patch a glyph into a font from another font

I have a monaco font patched for powerline that I downloaded from this repository. Now for licensing issue there are no free version of this font. Anyway, I use tmux which have U+2502 as a separator ...
0
votes
0answers
16 views

New entry in xmodmap creates new group instead of layer

Here is my dotfile: ! Key ! Shift+Key ! Mode_switch+Key ! Mode_switch+Shift+Key ! ISO_Level3_Shift+Key ! ISO_Level3_Shift+Shift+Key keycode 43 = h H BackSpace NoSymbol U20BD NoSymbol Cyrillic_er ...
0
votes
0answers
10 views

Some symbols are not rendered in Linux/Plasma5

I have Opensuse Tumbleweed with all the latest updates installed on my laptop and my PC, on my laptop with same font CURLY BRACKET SECTION symbols are rendered, but on my PC they are not, I see only ...
0
votes
1answer
24 views

UTF8String example for seagull

I am trying to test a diameter application using seagull. In the captured PCAP file I have 3GPP-SGSN-MCC-MNC, which is defined as UTF8String. In seagull, I defined UTF8String in base_cc.xml as: &...
1
vote
1answer
43 views

(Possible) inconsistent behavior of grep and less

I have a utf-8 file containing some Turkish text inside. (My system is MacOSX) $ file -I foo.merge $foo.merge: text/plain; charset=utf-8 When I try to see some Turkish specific characters by using ...
0
votes
1answer
48 views

Hebrew (Unicode) file-name support on CentOs 7

I have a VPS with CentOS 7 installed. I am trying to upload to the server a file that has a Hebrew name. I tried to upload it via FTP (Pure-ftpd) and I get: Failed to convert command to 8 bit charset. ...
0
votes
0answers
26 views

Fonts needed for browsers

I'm having this issue where many unicode symbols aren't displayed(just boxes instead) on websites. But aren't they supposed to be loaded(by the browser) from external sources? If not, what fonts do I ...
2
votes
1answer
44 views

Unicode in Gnuplot terminal

E.g., on the bash command line I can type a δ character using the Compose key and an ad hoc ~/.XCompose file, but if I try to do the same in Gnuplot what I get is 13:48 boffi@debian:~ $ δ bash: δ: ...
1
vote
1answer
42 views

Is there a fast tool to show all the unique unicode characters in a file and their count?

Does anybody know a tool to tabulate all the unique unicode characters and their counts in a file?
7
votes
3answers
2k views

Strange character in a file

I have an UTF-8 file that contains a strange character -- visible to me just as <96> This is how it appears on vi and how it appears on gedit and how it appears under LibreOffice and that ...
2
votes
1answer
46 views

UTF8 Character Makes File Inaccessible

If I run: scp me@example.com:/home/me/cömmön_file.jpg /home/me/ from my remote server I get: scp: /home/me/cömmön_file.jpg: No such file or directory If I swap out the utf8 characters though ...
2
votes
1answer
68 views

How to write a file list with foreign names to a csv file in the terminal?

I have some files with names in foreign languages in my terminal folder. On the terminal when I do 'ls' i see the list on the right in the picture below. When I do ls -t | tail -n +2, I see the real ...
0
votes
1answer
23 views

Remove non-alphas while ignoring certain unicode chars

I have a text like the following: []përgjithshme [" është "] 28.748 [< kilometra katrorë.>] This text contains cyrillic characters in the range U400 - U4FF. I want to strip out the non-...
1
vote
0answers
84 views

Unicode and Telegram Desktop app

Is there any way to enable unicode input in Telegram Desktop App for unix/linux systems? Addendm: The problem seems to be due to some qt-ibus configuration problem according to this unfortunately ...
4
votes
1answer
113 views

Is linux kernel implemented tty always render data using UTF-8?

It seems so. I read IUTF8, but the document only said it allows you to handle input as UTF-8 when doing line-editing. But what about output? If you use a GUI Terminal, it usually allows you to ...
7
votes
1answer
138 views

How to correctly display Polish diacritic symbols in groff?

I'm playing around with groff and I wanted to generate a pdf from the following test.ms .TL Tytuł .AU Imię Nazwisko .NH Wstęp .PP Pierwszy paragraf. Jakieś informacje, żeby były polskie znaki. .PP ...
1
vote
1answer
87 views

DWM Unicode Support

I'm in the process of creating a nice looking statusbar with DWM and I want to implement unicode symbols with font awesome. Supposedly I can use the Pango patch but its out of date (for version 6.1) ...
3
votes
1answer
256 views

SSH to FreeBSD in UTF-8

I'm trying to ssh to FreeBSD-11.2-Release from my Windows machine using Mintty, but when I set LC_ALL to en_US.UTF-8 and try to enter/paste a non-ASCII character, connection got closed. I'm using ...
35
votes
1answer
4k views

Should we use UTF-8 characters like ⏰ in bash/shell script?

The simple code here is working as expected on my machine if launched with bash : function ⏰(){ date } ⏰ Could there be a problem for other people using this, or is it universal ? I'm wondering ...
3
votes
1answer
93 views

Why is bash removing other digits?

On this (It is not intended to be a range, but an explicit list): $ a='0123456789 ٠١٢٣٤٥٦٧٨٩ ۰۱۲۳۴۵۶۷۸۹ ߀߁߂߃߄߅߆߇߈߉ ०१२३४५६७८९' $ echo "${a//[0123456789]}" ۰۱۲۳۴۵۶۷۸۹ ߀߁߂߃߄߅߆߇߈߉ ०१२३४५६७८९ Bash is ...
3
votes
1answer
117 views

Bash prompt wrap broken when path has unicode (Greek) characters

When a path is too long to fit on the terminal and it wraps, the following prompt is broken and you can't see properly what is typed. See screenshot below: These are the contents of .bashrc that set ...
25
votes
3answers
4k views

Why does sort say that ɛ = e?

ɛ ("Latin epsilon") is a letter used in certain African languages, usually to represent the vowel sound in English "bed". In Unicode it's encoded as U+025B, very distinct from everyday e. However, if ...
0
votes
1answer
165 views

Terminal shows non-ascii? characters in UTF-16 hex codes

todoroki@todoroki-VJZ13B ~>printf "ä\n" echo "ä" ä ä ä \udcc3\udca4: \u30b3\u30de\u30f3\u30c9\u304c\u898b\u3064\u304b\u308a\u307e\u305b\u3093 ...
3
votes
3answers
230 views

Using uniq on unicode text

I want to remove duplicate lines from a file with words of Syriac script. The source file has 3 lines, 1st and 3rd are identical. $ cat file.txt ܐܒܘܢ ܢܗܘܐ ܐܒܘܢ When I use sort and uniq, the result ...
0
votes
0answers
54 views

file command gives incorrect encoding type

I have a csv file in hand. When I ran 'file -i filename', it shows that it's encoded as us-ascii. But when I ran cat filename | csvcut -t -e us-ascii, I got an error: "Your file is not "us-ascii" ...
1
vote
2answers
57 views

Finding Greek from the command-line

How can I use perl on the command-line to print only lines containing Greek unicode characters? For example, only the second of the following lines hullo, world χαῖρε, ὦ κόσμε It should be ...
1
vote
1answer
219 views

Bash behaves weird when PS1 contains wide unicode characters

I'm trying to customize the bash prompt and here is my PROMPT_COMMAND prompt_command() { local time="${BLUE}\t${RESET}" local user="${GREEN}\u${RESET}" local hostname="${GREEN}\H${RESET}" ...
0
votes
0answers
49 views

How to check if `libc`, `font` and `terminal` all agree on Unicode version?

In my case, when I open certain files in tmux such as this file, and have multiple panes, the characters overflow the pane's boundaries. Nicholas Marriott suggested that the libc, font and terminal ...
1
vote
0answers
229 views

Chromium in raspbian is not loading fonts

I have flash project which has verdana/lohit font embedded into it but they do not load and shows boxes in Chromium (Chromium 65.0.3325.181) in raspbian. However same project gets loaded when ran in ...
0
votes
1answer
155 views

Filename displays with ? char when `ls` while with junk character when `ls | grep fran`?

What is the configuration that defines how file name with unrecognizable character be replaced with some other character. For example in my Linux system same file is displayed differently in UTF8 ...
1
vote
0answers
118 views

Onboard 1.4 crashes X.org on Arch Linux

Recently, a problem arose, which I suspect of being a bug, but I am not sure. I use Onboard virtual keyboard on my tablet PC powered by Arch Linux. It works nicely, but one thing is not working well. ...
0
votes
1answer
279 views

Wrong font for some unicode characters [closed]

I have been experiencing some issues with how my terminal shows some of the Unicode characters form my language. The picture shows the issue in the editor in the left terminal. It seems like some of ...
1
vote
0answers
133 views

refuse to cp/rsync/etc some files with strange characters

We have a Windows server with filenames containing unusual characters. For exemple picture.jpg (the unusual character is between the e and the .jpg). We mount it with mount -t cifs -o 'ro,...
1
vote
1answer
103 views

Can I write a console program that works with multiple character encodings? [duplicate]

I am writing a console program in C. I expect the Terminal that my program is running in to have its character encoding set to UTF-8. This means that I am sending UTF-8 encoded strings to the ...
0
votes
1answer
221 views

Bash (Yad) using floating point numbers instead of whole numbers [closed]

I wrote this bash script that runs multiple timers. The problem is my array stores whole numbers: |1|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|1 But the other user's LC_NUMERIC is setup as it_IT.UTF-8 and their ...
0
votes
1answer
85 views

Mutt and character sets issue

I updated to mutt 1.10.0 and now my character sets are all messed up. They worked fine with 1.5.21 -- system version and now old. Following the FAQ, I have these set up: ; echo $TERM ...
0
votes
2answers
47 views

How to only keep BMP in the utf-8 text file? [duplicate]

This is the content of a text file, and I want to only keep BMP. 这是无效的字符 𡛂 这是正常的字符 哈 In the How to remove non UTF-8 characters from text file , I find there is a solution, but it doesn't work ...
1
vote
1answer
60 views

Why do all standard commands include an ASCII letter?

Although one can easily make an executable file named ./42 or even ./カラオケ, I find no common packages that include letter-free commands. On Ubuntu, for example, apt-file find . | grep -i -e '/[^a-z/_\-...
2
votes
2answers
508 views

sed Match character range

Is there a way to match some Unicode range exactly. Let's use the Cyrillic range as an example: U+400 to U+52f The whole range of chars could be printed (from bash or zsh) with: $ echo -e $(printf '\...
0
votes
1answer
1k views

curl with Umlaut causes “JSON parse error: Invalid UTF-8 middle byte 0x22” [duplicate]

I'm running below curl command from the command line (Git Bash on Windows) or as part of a Bash Script. curl -s -X POST -H "Content-Type: application/json" -H "Accept: application/json" -d "{\"...
0
votes
1answer
43 views

What part of the Linux system is responsible for the effects of Ctrl+Shift+U?

If you press Ctrl+Shift+U on a Linux based system (not sure about UNIX) in any textbox you get a small underscored u. You can then proceed to enter hexadecimal numbers and confirm them with enter. The ...
6
votes
3answers
918 views

Replace UTF-8 characters with shell perl

How do I get perl to properly replace UTF-8 character from a shell? The examples use stdin, but I need something that works for perl ... file too. This is what I expect: $ echo ABCæøåDEF | perl -CS ...
1
vote
3answers
311 views

remove duplicated pattern/entries within each field in CSV file

How do I remove duplicated entries within each separate fields with below sample as data. 0x,9.4,,,#0,#UNIX#unix,#cli#L#فا#0#فا#0#L#SE#Cli#SE,#فارسی#فارسی#۱#1#١#1,bsh,#V & v expected output(...
4
votes
1answer
757 views

Making grep understand byte escapes

I'm trying to match against some UTF-8 characters. The problem is grep doesn't translate \x byte escapes, so this fails: echo -e '\xd8\xaa' | grep -P '\xd8\xaa' while this succeeds: echo -e '\xd8\...