The tag has no usage guidance.

learn more… | top users | synonyms

1
vote
2answers
876 views

How to convert unknown-8bit file to utf8

I have a .srt file that displays as gibberish when I open it in gEdit in ubuntu. So I want to convert it to utf8 to be able to read it. When I try to figure out what the encoding it give: file -i ...
2
votes
1answer
24 views

Encoding problem on aix system when logging using log4j2

I am using log4j2 to log message on aix system. In log when I use command less I can see: ...|LU?N ...|NOV�K ...|ZEM�NEK when I copy same file on my ubunty system and use same program I can see: ...
2
votes
2answers
810 views

view file containing DOS text (box-drawing characters, CRLF line terminators) and escape sequences

I have stumbled upon a strange text file in an archive of an old dos game downloaded from some abandonware site. The text file is named WHY_NOT.TXT. I assume the file is some sort of .NFO file from ...
5
votes
1answer
684 views

File name encoding wrong in KDE, but correct in GTK/tty1/

I'm on a new Linux (installed yesterday). When I access files from old partitions their file names have incorrect encoding (ä instead of ä, same for other umlauts) inside KDE apps like dolphin and ...
0
votes
1answer
2k views

AddDefaultCharset ISO-8859-1 not work

I have upgrade Apache from 2.2 to 2.4 and I am having problems with character encoding. In my pages I have è=è ò=ò ì=ì etc... In my httpd.conf I have commented the default value and added ...
18
votes
4answers
5k views

Understanding Unix file name encoding

I have a hard time understanding how the file name encoding works. On unix.SE I find contradicting explanations. File names are stored as characters To quote another answer: Several questions about ...
3
votes
0answers
18 views

Get consistent encoding for all files in directory

I have a directory containing lots of csv files from various vendors with two different encodings: ASCII Text / UTF-8 UCS2 / UTF-16 little endian I'd like to use grep, awk, sed and other utilities ...
3
votes
3answers
338 views

Convert UTF-8 code represented in terms of ASCII to UTF-8

I have a text file which is an ASCII file itself, but represents the code of utf-8: \350\207\252\345\212\250\346\216 Is there some program or command that can convert this ASCII file to a text ...
26
votes
3answers
15k views

What charset encoding is used for filenames and paths on Linux?

Does it depend on what file system I use? For example, ext2/ext3/ext4 but also what happens when I insert one of those "joliet" CD-ROMs with ISO 9660? I've heard that POSIX contains some sort of spec ...
0
votes
0answers
43 views

Nothing helps Gnome Terminal to not default to ANSI_x3.4-1968

I would really like gnome-terminal to default to a UTF-8 encoding, but whatever I do, I can't get it to do it. I don't have sudo access, so I cannot edit /etc/environment. I'm on a university-managed ...
26
votes
5answers
17k views

Filtering invalid utf8

I have a text file in an unknown or mixed encoding. I want to see the lines that contain a byte sequence that is not valid UTF-8 (by piping the text file into some program). Equivalently, I want to ...
1
vote
3answers
87 views

Why are file names surrounded wtih “â” sign?

After adding alias rm='rm -i' to my ~/.bashrc file, because when I remove file, shell wasn't asking for confirmation, file names are surrounded with "â" sign like in example below: rm: cannot remove ...
1
vote
1answer
173 views

Special characters in a web feed

I am running a script to check my gmail from the command line and display the first 35 characters in the title. curl -u username:password --silent "https://mail.google.com/mail/feed/atom" | \ grep ...
-1
votes
1answer
28 views

Usefulness of --from-code in command iconv

Is the --from-code option of the iconv command useful (http://man7.org/linux/man-pages/man3/iconv.3.html)? I mean, I don't see any difference if I specify this parameter (--from-code 'iso-8859-1' or ...
4
votes
1answer
47 views

Gedit displays text, but cat tells a different story

I am trying to sort a dictionary from Grady Ward's Moby and have encountered a bit of a problem. When I open it in gedit, I get: abaca×N abaciscus×N abacist×N aback×v abacli×p Abaco×N abactinal×A ...
0
votes
1answer
123 views

LMDE2 / MATE locale charset inconsistency

I have a machine with Linux Mint Debian Edition 2 "Betsy" (installed as RC, with all available updates applied) and MATE desktop environment. My problem is that in some applications special characters ...
1
vote
1answer
34 views

How to format password using SCP

I'm currently writing a Script to upload files to my server via SCP. I have the following line: sshpass -p password!% scp -P 1234 ./test.text user@mydyndns:~/test.txt But everytime I try that I ...
1
vote
0answers
15 views

Understanding script and font relation (character-map)

When I use charactermap, I found that if I change font, only Latin script changes. Character by all other scripts sounds independent with change in font! So, I want to know what is relationship ...
0
votes
0answers
19 views

Searching text data in Hebrew, Arabic or Aramaic

How can I search for a (sub-)string in a database that has several languages, including Hebrew? I am assuming that the text being right-to-left is a question of the presentation, but the logical ...
6
votes
2answers
2k views

Can vim display ASCII characters only, and treat other bytes as binary data?

I already know vim -b, however, depending on the locale used, it displays multi-byte characters (like UTF-8) as single letters. How can I ask vim to only display ASCII printable characters, and ...
1
vote
1answer
39 views

Can not get the file names with convmv

I have just uploaded some files on a FreeBSD server. Some files have the ??? into their names. The files came from a windows machine, so I try this: $ convmv -f cp850 -t utf-8 * Starting a dry run ...
2
votes
1answer
24 views

Cyrillic file names broken after upgrading Slackware

After upgrading glibc and some other libraries on Slackware instead of cyrillic names I see the garbage: | Was before | Is now | |------------+------------------| | Вождение | ...
4
votes
1answer
163 views

weird output when typing letter c in shell (non english input)

I had keyboard set to Georgian language input. Forgot to switch it back to English and typed letter c in shell. c = ც english c key is mapped to georgian letter ც - if this makes any difference. ...
28
votes
2answers
859 views

find(1): how is the star wildcard implemented for it to fail on some filenames?

In a file system where filenames are in UTF-8, I have a file with a faulty name; it is displayed as: D�sinstaller, actual name according to zsh: D$'\351'sinstaller, Latin1 for Désinstaller, itself a ...
2
votes
1answer
109 views

Convert TeX accents to UTF-8 in Python

edit below First of all, I am quite sure this is a duplicated question. But I have no idea how to search for it. I spent quite a long time already. I am reading with python from a .txt file the path ...
19
votes
5answers
3k views

Generating a random password; why isn't this portable?

I want to generate a random password, and am doing it like so: </dev/urandom tr -dc [:print:] | head -c 64 On my laptop, which runs Ubuntu, this produces only printable characters, as intended. ...
0
votes
1answer
54 views

make `tar` archive from NTFS with non-ascii file names

Loaded a live disk into a Windows based machine with NTFS files. Being a Linux user for some years, using tar came to my mind instantly and made some backup archives from partitions and stored them ...
4
votes
4answers
288 views

Make tail not modify the charset of the terminal

In a linux terminal (CentOS) I am using the command tail --follow=name my-rolling-file.log in order to see the logs of my application. Sometimes in the log there is some binary data dumped (I dump ...
10
votes
2answers
4k views

Convert a .docx to a .pdf with pandoc

I am trying to convert a .docx received by mail to a correct pdf by using pandoc (I am using GNU/Linux). I have an error concerning characters encoding : $ pandoc file.docx -o file.pdf pandoc: ...
0
votes
1answer
39 views

Error not found [closed]

I wrote this script to encode / encrypt data from a sql query by changing the names of tables and fields (I played on the ASCII character codes). #! /bin/bash chr() { [ "$1" -lt 256 ] || return 1 ...
0
votes
1answer
97 views

iconv can't convert circled digits to/from Japanese encodings

The unicode range of circled digits (U+2460 .. U+2468) cannot be converted to, or from, any of the Japanese encodings (EUC-JP, Shift-JIS, ISO-2022-JP), even though they exist there, and I run across ...
0
votes
2answers
283 views

Check whether text contains non-ASCII characters in a shell script

I have a file, I want to determine if it contains only English ASCII characters. I thought this is a very common question, but when I googled it, there is no direct answer or related. According to ...
5
votes
1answer
288 views

Greek characters in mutt's sidebar (mailboxes names sourced via offlineimap)

Mutt's sidebar, does not read properly Greek characters (mailboxes named by using Greek characters). Anyhow, there is no such problem inside both the index and the pager, where Greek ...
1
vote
4answers
261 views

How to recognize which ASCII character in hex is this?

We have a textfile that we want to clear from "bad" characters. If we open it with vim (with ":set number"): 57000044 zo¥<9a>¥ge¥o¥graph¥i¥cal¥ly 39999999 pariá¹<83>Å<9b>a The ...
1
vote
1answer
131 views

cp: cannot stat error - when filename has Asian characters

I am simply trying to copy files using cp -r /home/user/source/ /home/user/destination/ but it throws me the cp: cannot stat /source/filename.xxx error for some of the files. When I searched for this ...
0
votes
2answers
1k views

iconv cannot convert given characters

I just wanted to convert a txt file to UTF-8, since cat displays it correctly, but vi or gedit doesn't: $ cat test.txt ># >‹ | || ° ├── └── _ __ $ iconv -f WINDOWS-1253 -t UTF-8 test.txt ...
6
votes
2answers
4k views

tr complains of “Illegal byte sequence”

I'm brand new to UNIX and I am using Kirk McElhearn's "The Mac OS X Command Line" to teach myself some commands. I am attempting to use tr and grep so that I can search for text strings in a regular ...
2
votes
1answer
156 views

Question marks instead of national letters in console

I have Netgear Stora MS2000 (which uses a modified version of Red Hat) and have strange font issues while connecting by SSH. Note that this occurs both when connecting by PuTTY from Windows and by ...
1
vote
2answers
55 views

How to only leave in given characters in a file?

Purpose: tr -cd '\12\40-\176' but with accents, ex.: á, í, ű, ő, ö, ü, ű, ú, ó, é. In octal: "á" is 341 "\n" is 12 So I tried: [user@notebook ~]$ printf 'XXXá\nXXX' | tr -cd '\12\341' | ...
1
vote
1answer
67 views

How do I edit text in Cyrillic / Russian on OpenBSD?

I'm trying to make it possible to easily edit text in Cyrillic / Russian on a remote OpenBSD 5.2 amd64. I'm using emacs-21.4p18-no_x11, which doesn't show any Cyrillic characters whatsoever when I ...
1
vote
1answer
61 views

Any way to interpret pstree?

-bash-3.2$ pstree 29838 RBcron.shâââjavaâââ15*[{java}] -bash-3.2$ pstree 24423 bashââ¬âRBcron.shâââjavaâââ15*[{java}] ââvim -bash-3.2$ Any way to interpret this? Why would this happen?
2
votes
1answer
384 views

$Unix character set conversion

I'm confused by character-sets in Unix. I have a CSV file downloaded via SFTP: $ file -ib myfile text/plain; charset=us-ascii The purpose for this character-set quest is that the data within file ...
1
vote
0answers
105 views

Rename files with non-utf8 characters using detox

I am trying to use detox in the following way detox -s utf_8-only -r -v -n to detect invalid filenames on my system. The command performs mostly ok. Unfortunately it falsely detects also German ...
4
votes
2answers
212 views

Characters visible in vi, but not in cat.

I have a 300 lines file with ^@ character between each characters in the file. (I cannot post the entire contents for security reasons, so I am pasting only the first line) [mercury@app01 ftp_logs]$ ...
13
votes
1answer
420 views

Command to retrieve the list of characters in a given character class in the current locale

What could be way to retrieve a list of all the characters in a given character class (like blank, alpha, digit...) in the current locale. For instance, LC_ALL=en_GB.UTF-8 that-command blank ...
1
vote
1answer
188 views

script to parse integer and convert it to char code

I am noob on Unix. I've a very unique problem where and really appreciate your help. I need to parse string containing hash code value and convert the hash code is equivalent char representation. Here ...
0
votes
2answers
74 views

Find out which encoding method less uses to display the content of a text file?

There is a plain text file, which Emacs can't display correctly, but less can. I wonder if less automatically detects the encoding method for each individual text file, or just use a single default ...
3
votes
1answer
285 views

How do I view cp1251 text file in UTF-8 console?

Attempt 1: $ less subs.srt "subs.srt" may be a binary file. See it anyway? <C8><F2><E0><EB><FC><FF><ED> ... Attempt 2: $ LANG=ru_RU.CP1251 less subs.srt ...
1
vote
0answers
77 views

How do I configure medit to display UTF-8?

My environment is set to use UTF-8. I produce a file in UTF-8 encoding. It works correctly on the terminal. For example: $ echo ééé > test $ cat test $ echo é | od -t x1 0000000 c3 a9 0a ...
1
vote
1answer
192 views

iconv terminated by signal 7

I was running a little bash script on some text files: find . -name "*.dat" -exec iconv --from-code='UTF-8' --to-code='ASCII//TRANSLIT' {} --output={} \; My machine is a Ubuntu 14.04 LTS. After a ...