Questions tagged [unicode]
Unicode is a computing industry standard for the consistent encoding, representation and handling of text expressed in most of the world's writing systems.
470
questions
1
vote
0
answers
19
views
Pasting non-ascii (utf8) into remote urxvt terminal
For pasting text, in urxvt/rxvt-unicode one can use middle button to paste PRIMARY selection.
I can do such Mouse-Middle-Click paste in my local urxvt terminal and even a remote server, in Chinese/...
- 977
0
votes
0
answers
27
views
Script for awscli check not working with crontab schedule
I have written a small code snippet to check the aws cli version
#!/usr/bin/env bash
if [ -e "/usr/local/bin/aws" ];
then
myAWS="/usr/local/bin/aws"
else
...
- 1
8
votes
3
answers
1k
views
Box character doesn't display properly in Linux terminal
I was just writing a C++] program that uses the box characters to display information.
I ran the program on macOS and used the terminal app and it worked fine.
When I switched to Debian Linux using ...
- 137
2
votes
2
answers
33
views
How to combine settings from multiple locales in Linux?
When I installed Linux I set my locale to en_US.UTF-8. However I want to override some but not all of the settings in that locale. Specifically, I would like the Measurement to be Metric instead of ...
- 21
0
votes
1
answer
19
views
Command similar to ascii for ascii extended and/or for unicode?
ascii command in Linux is fast and great. It allows us to search for a character or for a code point and returns all relevant results for a given search. Is there something similar for ASCII extended (...
- 3
4
votes
1
answer
76
views
How do I create a zip that preserves unicode character composition on linux?
I'm on Debian. I have a file called Sóanr.jpg. According to https://emojidissector.com/, this is made of the following code points:
S 0053 LATIN CAPITAL LETTER S
o 006F LATIN SMALL LETTER O
...
- 143
1
vote
3
answers
74
views
Writing bash arguments with trunctation
I want to print the first two arguments of a bash function, with the unicode character \u2263 on each side using a two space separation. The thing is that the final unicode must display at column 70. ...
- 141
3
votes
1
answer
106
views
Different encoding/Unicode interpretation using terminal vs using shell script
I was working on a keymap script (map keys from one language keyboard layout to another). And after a lot of hard time trying to get everything working I found out that different characters are ...
- 111
3
votes
0
answers
53
views
Unnormalized UTF-8 directory names
I noticed something interesting in one of my directories:
$ ls -li
total 36
2625309 drwxrwxr-x 2 dotancohen dotancohen 4096 Jul 4 2022 Español
2625385 drwxrwxr-x 2 dotancohen dotancohen 4096 Jul ...
- 15.2k
0
votes
0
answers
29
views
Is there a way to remove specific emoji from being rendered in any application while using Cinnamon desktop?
I am slightly annoyed with some emojis.
So I was wondering, how could I remove/prevent some emojis from being rendered at all?
Replacing them with some other emoji like cute cat face could work too.
...
user556471
0
votes
1
answer
23
views
testdisk utility reports nonexistent files from a exFAT drive used with Windows - why?
I tried to recover lost files from an exFAT thumb drive with the testdisk package on linux. It was very good at finding deleted files. However as I went through the entries, I saw weird entries. The ...
- 1
1
vote
0
answers
24
views
Cannot use unicode shortcut on non-english layouts
I’m using US and RU layouts, and while I can use Ctrl+Shift+u, when I have US layout selected, when I try to use it with RU layout selected, it just doesn’t work. Didn’t find anything related to it in ...
- 19
1
vote
2
answers
82
views
Looking up and Inputing arbitrary unicode characters in console/terminal
I'm looking for a simple, generic way to input arbitrary unicode characters in a text document on the terminal(e.g. in a terminal editor).
A basic method I can imagine is having a simple text(utf-8) ...
- 179
3
votes
3
answers
208
views
My bangla keyboard layout does not match the physical keyboard
Steps:
Settings > Region & Language > Add Bangla to Input Sources
selected bn.
Here, is the gkbd-keyboard-display -l bn output.
In LibreOffice Writer, when I work with bangla fonts ...
- 1,976
0
votes
1
answer
50
views
Remap Keys to output Unicode characters
My goal is to typeset mathematics/physics more efficiently and more readable by remapping my keys to use the proper Unicode symbols.
Therefore, I want to remap my keys, such that pressing Alt Gr + ...
9
votes
1
answer
413
views
What value for LANG should I use for "sort -u“ correctly handle Chinese characters?
The context
Issues found
Issue no. 1
Issue no. 2
Issue no. 3
The question
The context
Today, I wanted to keep the unique lines in a file containing Chinese characters. I decided to use the sort ...
- 201
6
votes
1
answer
532
views
How to find directories with 4-symbol unicode characters eg `<0328>`
While backing up my directories, the ones with names containing Polish characters like ę, ą, ż, ź, etc got "duplicated".
When I do ls -al I get response like:
drwxrwxr-x+ 310 root users ...
- 193
2
votes
1
answer
87
views
Is my text mangled beyond repair?
My mangled Czech text:
NOTE ON CZECH BIRTH NUMBER VALIDATION IN CZECH LANGUAGE;
in Czechia birth number = personal identification number
========================================================
Do ...
- 26k
3
votes
1
answer
61
views
How to find (and rename) files with invalid characters (Latin1) on UTF-8 system?
I've got some old files from a Latin1 based system with names that are invalid on my UTF-8 system. They show as various codes or question marks. For example:
/tmp/test $ ls -1
'FRg_Pause lecture_'$'\...
- 2,060
1
vote
0
answers
58
views
XTerm: Bitmap font for ASCII, TrueType font for Unicode
I would like to display Chinese characters in the console, with TrueType fonts and normal characters in Bitmap fonts. So far, I have succeeded in displaying:
ASCII TrueType, Unicode TrueType (via ...
- 11
0
votes
0
answers
35
views
Build a reduced size libicudata shared library?
I need to build a reduced size libicu* shared library based on 67.1. I tried to follow their guides here and here. I ran ICU_DATA_FILTER_FILE=filters.hjson ./runConfigureICU Linux on Devuan ...
- 249
9
votes
1
answer
537
views
interpret \UXXXXXXXX Unicode codes in text file
I have the following file :
$ cat test
Villes visit\U000000e9es
How can I interpret those \UXXXXXXXX codes, e.g. how can I get :
$ cat test | pipe into something
Villes visitées
- 1,279
-1
votes
1
answer
496
views
How to change wsl 2 kali linux non-UNICODE terminal character encoding to UNICODE terminal character?
I am trying to use Airodump-ng in WSL 2 KALI LINUX and they give me the the following warninig:
Warning: Detected you are using a non-UNICODE terminal character encoding
when I type $locale charmap, I ...
3
votes
1
answer
295
views
used awk to search by unicode in txt file
How i can used awk to search in txt file by unicode like :
u+064C
u+064D
and please how i count results and print the number in first line.
- 33
1
vote
2
answers
122
views
Remove all characters from string, except specific multibyte range
How would one use sed or another tool to remove all characters except a specific multibyte range?
For instance, the range \xd8\xa0 - \xd9\x8a covers Arabic characters encoded in UTF-8. This was my ...
- 15.2k
1
vote
1
answer
181
views
How to add a compose character to a xkb keyboard layout
I created a custom Ukrainian ISO 9 layout successfully (by redacting /usr/share/X11/xkb/symbols/ua and several other files), it works and all but it lacks one letter, namely G with a grave i.e. 'G̀' (...
- 11
0
votes
1
answer
60
views
Typing unicode characters into ed
I wondered if it was possible to type Unicode characters like an em_dash and curly quotes into ed. I am completing a project that requires these special characters in the document.
- 440
2
votes
2
answers
428
views
replacement for tr with utf-8 capabilities
in order to isolate the last word in any line of a poem (to have a list of all rhymes), I put together several snippets of code obtaining this
awk '{print $NF}' input.txt | tr 'A-Z' 'a-z' | tr -sc 'a-...
- 329
0
votes
0
answers
100
views
Using UIM to insert Unicode input from keyboard
According to "Transparent" Unicode input from keyboard UIM supports entering Unicode characters with Ctrl+Shift+u like IBus, but without showing an interactive prompt. I haven't been able ...
- 1
2
votes
1
answer
329
views
VIM shows ^@ every other character and ^M^@ at the end of line
Sorry if it's answered somewhere, I have no idea how to look for it. I received a series of reports from a bank that I'm supposed to process and they seem to be... badly encoded?
First two lines in ...
- 995
2
votes
2
answers
139
views
How to write a plugin or script for Kate that takes a unicode code point and inserts the corresponding unicode character at cursor?
Entering arbitrary Unicode characters according to their code point is simple e.g. in Gedit (press Ctrl+Shift+u, enter the (hex) code point, press enter).
One can even use the terminal. For example, ...
- 161
0
votes
1
answer
629
views
Unable to change file encoding to utf-8
I modified a UTF-8 encoded xml file using vi editor and saved it.
I'm on Redhat Linux 7.9
I checked the file encoding after the changes and found it to be us-ascii
file --mime-encoding tmpfiles/08/...
- 417
1
vote
1
answer
371
views
Showing 𝐒, 𝐽, and 𝑁 in the terminal
How can I make 𝐒(U+1d412), 𝐽(U+1d43d), and 𝑁(U+1d441), which I can display clearly in my browser, show in the terminal?
I use Urxvt on Archlinux.
The ~/.Xdefaults file has this content:
URxvt*...
- 1,175
1
vote
2
answers
76
views
Disparity in rendering Malayalam glyph
Migrating from https://github.com/rsms/inter/issues/436
This is the correct rendering of the word prominence in Malayalam. പ്രാധാന്യം.
And this is how it renders on:
Arch Linux
Where
Screenshot
...
0
votes
1
answer
370
views
Suckless's st messes up UTF-8 sometimes
I wrote an echo program in Rust (which can deal with UTF-8) and occasionally st seems to mess up the UTF-8 input. I've tried different terminal emulators, like alacritty and gnome-terminal and they ...
- 121
0
votes
2
answers
486
views
How to remove all soft hyphens (U+00AD) from a file
There is a thing called Soft hyphen. Its unicode value is U+00AD and it cannot be seen in my text editor.
Apparently, a few of my files are riddled with it:
$ perl -C7 -ne 'for(split(//)){print ...
- 1,976
2
votes
1
answer
277
views
echo checkmark in shellscript (zsh) how-to
Why is this checkmark not printed correctly when executing my shellscript, even though echo alone outputs corrctly?
script:
#!/bin/sh
YELLOW='\033[1;33m'
NC='\033[0m' # no color
echo "enter your ...
- 23
3
votes
1
answer
424
views
What is "length" of a string in Bourne shell compatibles' `${#string}`?
Arising from this discussion:
When I have (zsh 5.8, bash 5.1.0)
var="ASCII"
echo "${var} has the length ${#var}, and is $(printf "%s" "$var"| wc -c) bytes long"
...
- 15.4k
0
votes
0
answers
289
views
UTF-8 Decoding And Fonts
I created an application that users write fancy texts. The only problem is that I don't know how to make command-line render this utf-8 charterer. It works well on every other charterers but it fails ...
- 101
0
votes
2
answers
1k
views
How do you type a number with a bar symbol over it?
100 divided by 3 equals 33.3 bar. How can I put a bar symbol over the three like this:
You'd think there would be 10 unicode characters dedicated to this, but I couldn't find them.
- 4,693
1
vote
1
answer
513
views
How can I enable Unicode support in buildroot?
I want to build a 32 bit x86 ISO image from buildroot that can display Unicode characters. How can I do that?
In make menuconfig, I've set BR2_ENABLE_LOCALE_WHITELIST="C en_US en_US.UTF-8"
...
0
votes
0
answers
194
views
Copying windows csv file data to Unix server file
I have this Unix server at work where resides a script which picks up a file on that same server and process each row. I believe we receive those files from data providers, which SFTP the files to us. ...
0
votes
0
answers
60
views
File with Multiple ISO encodings
I have an Arabic file with multiple ISO encodings; encoded in ISO-8859-6 and a few characters encoded in ISO-8859-1 (Because they don't have an equivalent in ISO-8859-6). I want to convert this whole ...
- 233
0
votes
1
answer
708
views
How does Linux display Unicode characters?
I have a simple Python PIL script that's render some list of unicode symbols with FreeSerif font, which is have the most full unicode support (about 65k glyph)
but in reality Arch Linux with last ...
- 53
0
votes
1
answer
355
views
No batch processing when: busybox find . -exec
I have access to only busybox.
I have a folder, like this:
prueba/$: ls
./uno
./dos
./tres
Each two subdirectories (uno, dos) have many files (with spaces in their names and non-ASCII characters) and ...
- 143
1
vote
2
answers
671
views
Can a file path be invalid UTF-8?
Title says it all: if I have the bytes of a file path and I try to decode them as UTF-8 is it possible that the operation will fail (i.e. the bytes of a valid unix file path can be invalid UTF-8)?
- 125
0
votes
1
answer
506
views
How do I make .bdf Format Bitmap Font work
I'm using an Arch-based Linux distro.
Recently I started a New Ricing for my BSPWM. This time I want to use a bitmap font.
Well I would like a font with glyphs, so I chose Siji.
Previously I used some ...
- 1
0
votes
1
answer
37
views
Accessing the UTF-8 Cuneiform Numbers and Punctuation Symbols
Does anybody know how to access the UTF-8 Cuneiform Numbers and Punctuation Symbols?
twash="\u12400" # two ash
thash="\u12401" # three ash
frash="\u12402&...
- 59
0
votes
1
answer
93
views
Using variables defined in other bash scripts [duplicate]
I have a bash script where I define UTF-8 Greek Symbols in a file named greek-utfb.sh. I want to run tests that display the variables for printing the greek letters in the file greek-utfb-scout.
--- ...
- 59
0
votes
2
answers
4k
views
How to print UTF-8 symbols
How can I print UTF-8 symbols on a terminal using bash commands.
This works
echo -e '\U2586'
But the following is failing
printf '%s\n' $(tput setaf 118) "\\u2586" $(tput sgr0)
- 59