Questions tagged [unicode]
Unicode is a computing industry standard for the consistent encoding, representation and handling of text expressed in most of the world's writing systems.
497
questions
0
votes
0
answers
51
views
Creating file name which contains non-ASCII Unicode character in C locale
What is best practice to create a file name, when run in the C locale, with data containing a non-ASCII Unicode character?
For example, the data for the file name is 9 She\342\200\231s My Girl.mp3 (...
- 875
0
votes
0
answers
45
views
mailx doesn't decode Content-Type: text/plain; charset="utf-8"
I'm sending mail from python and am trying to read them on the console with mailx. As long as the body of the mail only contains ascii characters the mail is send with charset="ascii" and is ...
2
votes
1
answer
651
views
Which interpreter for "Unicode text, UTF-8 text executable"
I'm trying to set up a keybinding for an executable which is in my home. For this, I set the command:
sh -c '\"/path/to/the/executable\" --options'
But, it does not work, and, when I'm ...
- 463
11
votes
3
answers
2k
views
UTF-8 characters in POSIX shell script *comments* - anything against it?
I would like to include a couple of non-ASCII characters in my POSIX shell script comments. Note this is in no way a duplicate of e.g. "Which character encodings are supported by posix?" as ...
- 29.3k
0
votes
0
answers
18
views
Kernel support for displaying fonts with rotations and reflections (with or without Unicode support)
I am posting to this group because I was wondering whether the Linux kernel can be extended to support the feature of displaying fonts with reflections and rotations, whether these are stored as part ...
1
vote
0
answers
35
views
Ignore Accent Differences in Zsh Autocomplete
Suppose I have a directory named cálculo in the current directory. How can I autocomplete its name after typing the starting characters without the accent?
$ cd calc<tab>
$ cd cálculo/
I failed ...
- 207
1
vote
1
answer
76
views
Fontawesome icons are not pasted correctly
I am using Fedora and installed fontawesome via sudo dnf install fontawesome fonts. Later because it didn't work I also additionally installed the font manually via downloading the zip from the Github ...
- 23
2
votes
0
answers
79
views
How do I disable UTF-8 in an xterm (or X, really)?
I have a system running Debian unstable where I don't want to have UTF-8 in my xterms (or at all). But I recently discovered that somehow I now have UTF-8 in my xterms and other windows. It might have ...
- 21
0
votes
0
answers
47
views
ls: single-column vs. multi-column layout, non-Unicode characters in filenames
Create a directory ~/test with abcdefghijklmnopqrstuvwxyz and zyxwvutsrqponmlkjihgfedcba files in it.
ls ~/test will list them using multi-column layout:
abcdefghijklmnopqrstuvwxyz ...
- 499
2
votes
2
answers
140
views
Search and replace composed Unicode characters
I have a deep folder structure on a Debian machine
where the directory names and the filenames
contain some "special" characters (ä,ö,ü).
However, these are not in "ISO-8859-1"
...
- 769
3
votes
1
answer
108
views
'ls name' and 'ls | grep name' with accent different
I am on Xigmanas (NAS freebsd). I'll explain the situation as simply as possible:
:; set | egrep 'LC_A|LANG'
GDM_LANG=fr_FR.UTF-8
LANG=fr_FR.UTF-8
LC_ALL=fr_FR.UTF-8
SLIM_LANG=fr_FR.UTF-8
:; ls -i ...
0
votes
0
answers
62
views
Terminal: Help understanding behavior with UTF-8 text
I am trying to understand the following behavior I am observing on my Ubuntu system. Consider the following two files:
$ hexdump -C 1.txt
00000000 d9 82 d8 a8 d8 a7 d9 86 d9 8a 5e d9 84 d9 86 d8 |.....
- 3,142
1
vote
0
answers
22
views
XQuartz xterm UTF-8 resource name
I was using UTF-8 resources names like these ones:
wengé*Background: #321
wengé*Foreground: #ffb
and this was working with XQuartz 2.8.1 through this direct
call like from within the ...
- 1,000
0
votes
0
answers
118
views
Make GNOME Characters use different emoji font?
How do I configure which emoji font GNOME Character app uses? Currently it's using Noto Emoji Color.
GNOME 45.3
Arch Linux
- 175
1
vote
1
answer
35
views
Crossmark symbol (\u274c) doesn't work in debian 12
I have moved from Ubuntu 22.04 to Debian 12, I have a bash function that outputs crossmark if command failed and checkmark if command succeed. The checkmark works, but the crossmark doesn't.
Here is ...
- 131
0
votes
1
answer
179
views
What puts the terminal in Unicode mode?
I have a Debian server which is not properly displaying Unicode characters when logged in locally, without starting X11. Unicode works after running unicode_start (until the terminal is closed). It ...
- 153
1
vote
1
answer
39
views
How to use unix `mv` to rename files with unicode spaces(not U+20)?
$ ls cn*
cn blah blah.txt
$ ls cn\ *
ls: cannot access 'cn *': No such file or directory
$ ls cn*|hexdump -C
00000000 63 6e e2 80 85 62 6c 61 68 c2 a0 62 6c 61 68 2e |cn...blah..blah.|
00000010 74 ...
- 13
4
votes
2
answers
197
views
Why is ls sorting Chinese filenames by length?
I've run into a bit of a weird behaviour that I don't fully understand with ls and Chinese filenames. I'm running macOS 13.6.1 with SIP enabled (no core OS modifications), MacPorts installed, and US ...
- 1,178
2
votes
1
answer
447
views
Can awk be told to count the character string length rather than byte string length for '%10s' printf formats?
Try this for an output of |Ü| X|:
echo 'Ü X' | awk '{printf("|% 2s|% 2s|\n", $1, $2)}'
Obviously awk counts the byte length, not the character length of the Ü, so the count is 2 and no left ...
- 958
0
votes
2
answers
100
views
groff -mandoc creating "ESC[1m" versus overstriking with backspace for bold text
I found that groff uses different ways to indicate bold text for the utf8 output format.
On FreeBSD 14, groff emits escape codes for a terminal (ESC, [1m):
$ printf ".Dd today\n.Sh NAME\n" | ...
- 1,787
0
votes
1
answer
116
views
Why is MB_CUR_MAX 6 instead of 4 for UTF-8? (Linux, glibc)
MB_CUR_MAX is defined by glibc as 'a positive integer expression that is the maximum number of bytes in a multibyte character in the current locale.'
If I print the value I get 1. I assume that this ...
- 254
0
votes
2
answers
224
views
I need to create a pipe to convert string from UTF-8 to UTF-7-IMAP
To automate the command line creation of hundreds of directories in IMAP maildirs, I would need to be able to convert UTF-8 strings to UTF-7-IMAP on the fly.
In php, I found a way to do it with a ...
- 43
0
votes
2
answers
167
views
Listing filenames with special characters
I have a zsh shell (with oh-my-zsh default config). Why I ls filenames with special characters, they are printed as:
''$'\316\262''=0.35-L=32-m=10.jld2'
This should be:
β=0.35-L=32-m=10.jld2
but the ...
- 1,757
1
vote
0
answers
43
views
Debian terminal not displaying correct Unicode half-block characters [duplicate]
I have a program that prints Unicode half-block characters (U+2580, U+2584), but on Debian 10 terminals (just the fullscreen terminal, no X), it's printing diamonds instead of half-blocks.
The two ...
- 1,515
1
vote
0
answers
74
views
Ctrl-Shift-U requires *extra* U in Ubuntu 23.04 Cinnamon?
I'm running a new install of Ubuntu 23.04 with cinnamon desktop 5.6.7
Typing Ctrl-Shift-u in a terminal does nothing unles the next character is another u; then the underlined u appears and I can ...
- 121
0
votes
2
answers
731
views
Entering special characters the same way on Windows and Linux
ctrlshiftu followed by the hex value of a Unicode character enters that character. For example, ctrlshiftu41 enters 'A', whose value is 0x41 in hex and 65 in decimal.
There's also the compose key, ...
- 388
1
vote
1
answer
78
views
Expand tabs in file with utf8 characters
I use expand to expand tabs to spaces. For utf8 files expand doesn't work correctly. E.g. in ć\ta tab is expanded to 6 spaces while in a\ta to 7 spaces.
How do I make it work for utf8 files?
- 253
-1
votes
2
answers
74
views
Is ∞ allowed in UTF-8 Encoded files?
Are lemniscates, ∞, allowed in UTF-8 Encoded files?
I am hoping that students with less than six months of computer programming experience can use a search engine to type something like "is ...
2
votes
2
answers
149
views
Unicode Supplementary Multilingual Plane (Plane 1) glyphs in xterm
I'm trying to display Unicode Supplementary Multilingual Plane (Plane 1) glyphs in xterm. Those glyphs are in the U+010000..U+01FFFF range (https://unifoundry.com/pub/unifont/unifont-15.0.01/...
- 55
7
votes
1
answer
2k
views
How should I interpret the fact that a Unicode code point is shown in two completely different ways in two different terminal emulators?
This is kind of a spin off from an older question I asked.
Here's the screenshot from that question:
In the bottom left is URxvt, and you can see a lighting bolt-like icon at the beginning of the ...
- 1,687
4
votes
4
answers
366
views
Collect chars from strings and print their unicode
Context (skip, if you don't care; read, if you suspect I'm totally on the wrong track)
For an embedded system with small memory, I want to generate fonts which contain only those glyphs actually ...
- 13.6k
1
vote
0
answers
97
views
Pasting non-ascii (utf8) into remote urxvt terminal
For pasting text, in urxvt/rxvt-unicode one can use middle button to paste PRIMARY selection.
I can do such Mouse-Middle-Click paste in my local urxvt terminal and even a remote server, in Chinese/...
- 1,694
0
votes
0
answers
219
views
Script for awscli check not working with crontab schedule
I have written a small code snippet to check the aws cli version
#!/usr/bin/env bash
if [ -e "/usr/local/bin/aws" ];
then
myAWS="/usr/local/bin/aws"
else
...
- 1
8
votes
3
answers
3k
views
Box character doesn't display properly in Linux terminal
I was just writing a C++] program that uses the box characters to display information.
I ran the program on macOS and used the terminal app and it worked fine.
When I switched to Debian Linux using ...
- 137
2
votes
2
answers
614
views
How to combine settings from multiple locales in Linux?
When I installed Linux I set my locale to en_US.UTF-8. However I want to override some but not all of the settings in that locale. Specifically, I would like the Measurement to be Metric instead of ...
- 21
1
vote
1
answer
76
views
Command similar to ascii for ascii extended and/or for unicode?
ascii command in Linux is fast and great. It allows us to search for a character or for a code point and returns all relevant results for a given search. Is there something similar for ASCII extended (...
- 13
4
votes
1
answer
252
views
How do I create a zip that preserves unicode character composition on linux?
I'm on Debian. I have a file called Sóanr.jpg. According to https://emojidissector.com/, this is made of the following code points:
S 0053 LATIN CAPITAL LETTER S
o 006F LATIN SMALL LETTER O
...
- 143
1
vote
3
answers
83
views
Writing bash arguments with trunctation
I want to print the first two arguments of a bash function, with the unicode character \u2263 on each side using a two space separation. The thing is that the final unicode must display at column 70. ...
- 1
3
votes
1
answer
278
views
Different encoding/Unicode interpretation using terminal vs using shell script
I was working on a keymap script (map keys from one language keyboard layout to another). And after a lot of hard time trying to get everything working I found out that different characters are ...
- 151
3
votes
0
answers
184
views
Unnormalized UTF-8 directory names
I noticed something interesting in one of my directories:
$ ls -li
total 36
2625309 drwxrwxr-x 2 dotancohen dotancohen 4096 Jul 4 2022 Español
2625385 drwxrwxr-x 2 dotancohen dotancohen 4096 Jul ...
- 16.1k
0
votes
0
answers
122
views
Is there a way to remove specific emoji from being rendered in any application while using Cinnamon desktop?
I am slightly annoyed with some emojis.
So I was wondering, how could I remove/prevent some emojis from being rendered at all?
Replacing them with some other emoji like cute cat face could work too.
...
user556471
0
votes
1
answer
76
views
testdisk utility reports nonexistent files from a exFAT drive used with Windows - why?
I tried to recover lost files from an exFAT thumb drive with the testdisk package on linux. It was very good at finding deleted files. However as I went through the entries, I saw weird entries. The ...
- 1
1
vote
0
answers
37
views
Cannot use unicode shortcut on non-english layouts
I’m using US and RU layouts, and while I can use Ctrl+Shift+u, when I have US layout selected, when I try to use it with RU layout selected, it just doesn’t work. Didn’t find anything related to it in ...
- 19
1
vote
2
answers
550
views
Looking up and Inputing arbitrary unicode characters in console/terminal
I'm looking for a simple, generic way to input arbitrary unicode characters in a text document on the terminal(e.g. in a terminal editor).
A basic method I can imagine is having a simple text(utf-8) ...
- 181
3
votes
3
answers
507
views
My bangla keyboard layout does not match the physical keyboard
Steps:
Settings > Region & Language > Add Bangla to Input Sources
selected bn.
Here, is the gkbd-keyboard-display -l bn output.
In LibreOffice Writer, when I work with bangla fonts ...
- 2,748
1
vote
1
answer
522
views
Remap Keys to output Unicode characters
My goal is to typeset mathematics/physics more efficiently and more readable by remapping my keys to use the proper Unicode symbols.
Therefore, I want to remap my keys, such that pressing Alt Gr + ...
- 23
10
votes
1
answer
664
views
What value for LANG should I use for "sort -u“ correctly handle Chinese characters?
The context
Issues found
Issue no. 1
Issue no. 2
Issue no. 3
The question
The context
Today, I wanted to keep the unique lines in a file containing Chinese characters. I decided to use the sort ...
- 211
6
votes
1
answer
623
views
How to find directories with 4-symbol unicode characters eg `<0328>`
While backing up my directories, the ones with names containing Polish characters like ę, ą, ż, ź, etc got "duplicated".
When I do ls -al I get response like:
drwxrwxr-x+ 310 root users ...
- 193
2
votes
1
answer
111
views
Is my text mangled beyond repair?
My mangled Czech text:
NOTE ON CZECH BIRTH NUMBER VALIDATION IN CZECH LANGUAGE;
in Czechia birth number = personal identification number
========================================================
Do ...
- 29.3k
4
votes
1
answer
696
views
How to find (and rename) files with invalid characters (Latin1) on UTF-8 system?
I've got some old files from a Latin1 based system with names that are invalid on my UTF-8 system. They show as various codes or question marks. For example:
/tmp/test $ ls -1
'FRg_Pause lecture_'$'\...
- 2,150