Questions tagged [unicode]

Unicode is a computing industry standard for the consistent encoding, representation and handling of text expressed in most of the world's writing systems.

Filter by
Sorted by
Tagged with
2
votes
1answer
33 views

Mutt subject is garbled when using japanese/chinese characters

When using mutt command: mutt -e "set content_type=text/plain" -e "my_hdr From:abc@gmail.com" -s "取样更多 サンプル 样品 text" -c "取样更多" -a "サンプル取样更多" Now ...
2
votes
3answers
65 views

Padding unicode strings with bash's printf

I tried padding a Unicode string with bash's printf, and saw that, while printf "%2s" a yields the expected " a", a Unicode variant printf "%2s" ä yields a ...
0
votes
0answers
15 views

Changing font in mlterm

I am not able to see complex fonts like Devanagari, Arabic, etc correctly in terminal-- they do render but circles appear on the screen for Devanagari, Arabic is also garbled. Everything is fine in ...
6
votes
1answer
195 views

Is the `utf8` in `en_US.utf8` a canonical character set?

The output of locale seems to distinguish between upper and lowercase: % locale -a C en_AU.utf8 en_US.utf8 POSIX More commonly, I've seen the hyphenated and uppercase UTF-8. What is the canonical ...
0
votes
0answers
16 views

How to echo a lozenge using bash?

I want to provide output like zsh powerline does: My main goal is to output a lozenge indicating an active item in rofi which I'll be using as a dmenu. How can I echo "Active" as a lozenge ...
1
vote
3answers
35 views

How do I remove Chinese characters from string B whenever they occur in string A, while preserving the order?

I have some long strings of Chinese characters, but for the sake of the question, let's suppose they are: 啊爱好情人心安静排全装按照八把握爸吧白菜酒色夭百班长板 and 阿姨啊挨打矮小爱国护安检慰置岸边上按摩时案子暗示巴士拔 The first string lists all of ...
0
votes
1answer
40 views

How iconv and od handle endianness? [closed]

echo hello | iconv -f ascii -t utf-16le | od -x produces what seems to me like a big-endian result 0068 0065 006c 006c 006f 000a whereas the same line without the 'le' produces, on a ...
1
vote
0answers
48 views

Why does urxvt display this box drawing glyph differently?

I'm switching to urxvt and noticed something on the way. I use zsh and powerlevel10k with it. The powerline, if that's what it's called, is being differently on urxvt, even though both konsole and ...
3
votes
1answer
30 views

zsh: indic fonts support rendering issue which is working fine on bash

I am using gnome-terminal 3.36 version In bash I can type without issues $ अखिल् In zsh it's not showing properly $ अखिल<094d> this is zsh behaviour for indic fonts which are working fine ...
2
votes
2answers
95 views

How widespread is the C.UTF-8 locale?

As far as I know, the C.UTF-8 locale started as Debian's attempt to modernize the standard C locale and over time some other distros, such as Fedora, added support for it. But what about other Linux ...
0
votes
0answers
54 views

Question marks in my kali linux shell

I am trying to do active reconnaissance in Kali Linux (4.19.0-kali3-amd64) using xprobe2 tool. So I typed the command xprobe2 espn.com (for example). I know that espn.com use "OpenBSD" ...
3
votes
1answer
118 views

Can I make shell consider NBSP a whitespace character?

This seems like a common issue: you paste a simple command from somewhere into the terminal and it's not being understood by a shell: $ echo test $ zsh: command not found: echo test The reason is ...
0
votes
1answer
47 views

What is the encoding format of a binary file(OBJ file,a.out,ELF etc..) [closed]

When I open a binary file in vim, I see the mnemonics instead of the 0s and 1s. I believe this is due to the fact that vim by default opens the file in ASCII mode. Is my understanding right? What is ...
2
votes
1answer
103 views

How can I make a custom terminal cursor shape?

I want to change the shape of the cursor in my various (emulated) terminals. The shape that I want is ⼕ (sorry if it doesn't render). It's a three sided box that opens to the right. This way I could ...
-1
votes
2answers
70 views

Determine missing font for a specific unicode character

There are symbols from this table, which are not rendered on my system; as in, they're just rendered as vertical rectangles to mean I do not have the font to display them. This is the case, for ...
2
votes
1answer
47 views

How can I examine the Unicode encoding of a text document

Given a text document or string in an encoding such as UTF-8, what is the easiest way to find out what Unicode encoding represents the characters I see? I should like to find out which code points ...
0
votes
1answer
45 views

how to copy files without breaking utf8 filenames in ubuntu 8.04?

I have a very old PC which is unable to run pretty much anything. It previously used Windows 7, but now everything I try just makes it reboot. I, finally, managed to get access to it by running ...
5
votes
2answers
315 views

How to handle “YouTube filenames” containing arabic, chinese, hindi characters in bash script?

I downloaded many YouTube videos and want to process them using bash scripts. However the filenames used contain all kinds of special and non-ASCII characters. How do I handle this in a bash script? ...
0
votes
1answer
32 views

Dead keys - combining functionality -inconsistent

I don't completely understand dead keys. The dead acute for example always behaves in the following way: You have to know which character is compatible and then you can type that character with an ...
6
votes
1answer
251 views

sort command adds/removes data at will

I'm using sort command to sort rockyou.txt wordlist, which I downloaded from this site: % sort rockyou.txt > rockyou_sorted.txt However, when I then check the file sizes of both files, they ...
9
votes
1answer
971 views

Unix sort does not accept the unconventional delimiter '±' as a field separator

I tried both ... | sort -k'1,1r' -k'2,2' -t'±' ... | sort -k'1,1r' -k'2,2' -t$'\xC2\xB1' *$> sort: ±: Invalid argument* I hope it is not unsupported by sort, but just that I am missing some ...
0
votes
1answer
276 views

Can a unicode character be used for the zsh prompt instead of the '%' symbol?

Macbooks now default to zsh for new users. Existing users have the option of switching to it. I've made the switch and would like to make my prompt look the way it did in bash. I have this for the ...
3
votes
2answers
245 views

trying to use printf to decode unicode characters passed in as arguments

I am trying to printf some unicode codes that I pipe in like this echo 0024 0025 | xargs -n1 echo # one code per line | xargs printf '\u%s\n' hoping to get this $ % but this is what I get ...
0
votes
0answers
884 views

How to define fallback fonts for XTerm

I'm on a fresh Debian Buster installation with packages fonts-noto-core and fonts-symbola. I want to use "Noto Sans Mono" as the main font for XTerm, so I set the following line with xrdb: xterm*...
0
votes
1answer
62 views

Why is sed not doing what is defined in the locale collation file for grep (and sed, awk)?

Having a file with all the (printable) ascii characters: $ printf '%b' "$(printf '\\U%x\n' {32..126})" > file That could be sorted (using tr to reduce the long output to one line): $ sort file | ...
1
vote
0answers
106 views

zsh some unicode input shown as code in angle brackets in prompt line (but not output)

In my cygwin / mintty setup, when I have the unicode characters and U+1F60C (😌 relieved face) in my Windows clipboard, type echo ', paste, type ' and enter, zsh displays it as inverse video <d83d&...
0
votes
1answer
47 views

Unicode character in PS1 makes wrapping behave weirdly (not a duplicate)

I wanted to put the Unicode character Δ into PS1, but when I did, when typing in long commands, the terminal wouldn't wrap to a newline but would overwrite the current line! The problem doesn't occur ...
3
votes
0answers
45 views

Is this a bug in 'tr'? [duplicate]

My terminal is set to UTF-8. If I type this: echo -n 'ýá' | xxd I can see this output: 00000000: c3bd c3a1 Which is fine. Now I would like to remove the 'ý' character from the string, so I use: ...
0
votes
1answer
63 views

How can I type Russian characters with accents in KDE?

I use KDE and have enabled the Russian keyboard layout. This works fine except that I see no opportunity to add accents to vowels. Is this in fact possible? I wish to use them in applications such as ...
0
votes
0answers
28 views

Xterm does not display unicode character [duplicate]

Fellow people of the internet, I got an issue with Xterm, as i try to use a zsh theme using unicode characters. However, Xterm refuses to print them, which is a bit annoying. My Xressources is a ...
0
votes
1answer
53 views

How to turn mojibake text to readable form?

I got mojibake text (from MP3 ID3 fields) that need to be converted to the right form. I tried to: echo 'c2 8d 44 e2 80 99 c2 b2 c2 8d 44 e2 80 99 c2 b2' | xxd -p -r | iconv -f sjis -t utf8; echo ...
-3
votes
2answers
177 views

Proposed additional POSIX “Character Classes” [closed]

There are a handful of "character classes" defined in POSIX as in LC_CTYPE locale definition with the following (12) names: alnum alpha blank cntrl digit graph lower print punct space upper xdigit ...
0
votes
1answer
19 views

How to take accent in account on centos [6|7] in bash

I'll give you an exemple, it would help. on my ubuntu LTS 18.04 touch /tmp/"Éléments supprimés" ls -l /tmp/*suppr* -rw-rw-r-- 1 USER USER 0 mars 4 08:28 '/tmp/Éléments supprimés' on my centos 6 ...
0
votes
1answer
67 views

Replace character in file based on position

I work on the encoding of a file, from unknown-8bit to UTF-8. but I got an error message. iconv: illegal input sequence at position 136828 So I did a search in the file with the position like this : ...
0
votes
2answers
512 views

How to fix strange characters in my .txt file?

I edited a .txt file in Visual Studio on Windows and then copied that into a HPC server. The file looks fine to me at first, But when I opened it in the linux environment, weird characters showed up (...
4
votes
1answer
107 views

weird unicode behaviour

I'm using Arch Linux and one thing I just can't seem to get right is the Unicode fonts. I'll give you an example of what goes wrong. Here's a screenshot of gucharmap: The selected glyph in the top ...
2
votes
0answers
48 views

Why did bash (or PuTTY) interpret the two characters 'cd' as the single character '㏅'?

I ssh'd into a CentOS VM today through PuTTY -- right after I logged in and got a bash shell, I tried to cd to another directory, but the displaying of what I typed lagged a bit, and then instead of '...
0
votes
1answer
62 views

stop bash from printing invalid unicode sequences

Doing echo -e "\uDDAA" (which is not a valid utf-8 codepoint) in bash, prints ���. How can I make it not print anything if its not a valid codepoint? what im trying to do is add in front of all ...
8
votes
1answer
881 views

Sort LC_ALL=C vs LC_ALL=C.utf8

Does the linux sort command differentiate between C and C.utf-8 locale? the sort manual says to use LC_ALL=C to sort by byte value, but i saw that C.utf-8 allows also utf8 values (and not just ASCII) ...
1
vote
3answers
108 views

How to insert unicode to file

I'm using fish-shell on CentOS8. I would like to insert the following to file. function fish_title echo "🔥 AmericaN 🔥" end But upon trying to paste it into the file, I see the following: ...
0
votes
5answers
928 views

How to draw a continuous line in terminal?

I've been searching for this for a while and didn't get a satisfactory answer for it. I've taken this screenshot from the output of Gnu/Linux's tree command. I want to know how to draw or print ...
2
votes
0answers
83 views

rxvt-unicode-256color (Debian, 9.22-6): braille characters only shown after showing arabic character

I'm trying to display braille characters in my urxvt terminal, e.g. I'd like to see a dot when running echo -e "\u2840". To my confusion, this only works if I run echo -e "\u0641" first in that ...
2
votes
1answer
458 views

Extract mail header and decode MIME encoded-word

As part of a script, I want to take a file containing an email and extract a header, e.g., the subject header. The following almost works: formail -c -x subject < FILE But it does not decode ...
0
votes
1answer
25 views

Cedilla does not work under Softmaker Office

Infos here Cedilla works in all other apps but SoftMaker office where it delivers a ć. What might I be missing? This might be some QT/GTK issue but I'm not sure about how to fix.
0
votes
2answers
189 views

Unicode Strings in Text file

I have a text file containing the following unicode strings with regular text. Cat a.txt {"relationship":{"type:Memberkey","id""824-\u0001\u0019BFGHDICA2166-01-01","source"} Here \u0001 and \u0019 ...
2
votes
2answers
191 views

Get the Unicode code point of a character in zsh

How can I get the Unicode code point of a character in zsh? e.g., if I have: s=abc How can I get the code point of s[2] (i.e. code point of "b" = 0x0062)?
1
vote
0answers
38 views

Why two unicode-characters are not allowed in xkb?

I wonder why a sequence of Unicode-characters is not allowed on one key-press in xkb. Eg. I use the Indian keyboard for typing in Devanagari. This keyboard is based on the inscript keyboard layout. In ...
3
votes
1answer
359 views

Can't use different font families in groff -ms

I'm using groff with ms macros and outputting it to PostScript. On Linux Void. I want to have a different font and also output Unicode characters. I have converted my font as written in this question. ...
6
votes
2answers
143 views

Xterm does not display one uni-code character

My xterm supports uni-code. For instance, it displays the Euro sign: echo -e '\xe2\x82\xac' But it does not display one particular character: PL_BRANCH=$'\ue0a0' echo $PL_BRANCH This character ...
2
votes
1answer
857 views

How to use a string in Chinese characters as input for another command in bash?

I'm currently using a command to convert some Chinese characters into pinyin, which requires the string in my bash to be Unicode safe and put the result in another variable. I can run the following ...

1
2 3 4 5
8