The tag has no wiki summary.

learn more… | top users | synonyms

1
vote
3answers
6k views

How to change encoding from Non-ISO extended-ASCII text, with CRLF line terminators to UTF-8?

I have a txt file : $ file -i x.txt x.txt: text/plain; charset=unknown-8bit $ file x.txt x.txt: Non-ISO extended-ASCII text, with CRLF line terminators And there are some characters that are ...
1
vote
2answers
68 views

Incorrect output from cp, rm, and so on

When I run, say, cp, I get output like the following: # cp -v Foo Bar âFooâ -> âBarâ What's up with the weird â characters? Why is the shell doing this? It looks like some kind of strange ...
2
votes
0answers
44 views

Character encoding of .netrc

Does anyone know which character encoding is used in .netrc? As a German user, I sometimes put umlauts in passwords. So my question is, are these treated as latin1 or may I assume that I am doing ...
0
votes
3answers
826 views

echo bytes to a file

I'm trying to connect my rasberry Pi to some display using the i2c bus. To get started I wanted to manually write stuff, bytes in particular to a file. How do you write specific bytes to a file? I ...
1
vote
0answers
141 views

Character encoding issue with my linux install?

I just installed Ubuntu 12.04.4 64 bits, all seemed to be ok, when I started to work on console I noticed: that I couldn't get some accented characters typed in to the console, for example, if I ...
1
vote
1answer
32 views

Why does SLES 11 corrupts given characters?

If I open a notepad and copy this from it: aña, to a file opened with vi on the machine, I get this: aña My question: Why? What could the problem be? Logfiles on the server has the same ...
1
vote
1answer
43 views

Why doesn't `restore` display my UTF-8 encoded filenames correctly in interactive mode?

When using the interactive mode of the restore utility to restore backups made with dump UTF-8-encoded filenames doesn't display correctly, see example below. The ??s should be ös ... $ sudo restore ...
3
votes
2answers
277 views

MP3 tags Cyrillic chars

I have some mp3 files with Cyrillic chars on file names and also on their tags. I'm using audacious to play them. See the file info below: Is it possible to change some encoding to show the ...
2
votes
3answers
74 views

Which terminal encodings are default on Linux, and which are most common?

I need to make a decision regarding whether a complicated commercial program that I work on should assume a particular terminal encoding for Linux, or instead read it from the terminal (and if so, ...
9
votes
3answers
2k views

identify files with non-ASCII or non-printable characters in file name

In a directory size 80GB with approx 700,000 files, there are some file names with non-English characters in the file name. Other than trawling through the file list laboriously is there: An easy ...
4
votes
1answer
119 views

login with xdm and umlaut in password

I want use xdm to login to my machine and start my session. My password contains an umlaut. I'd rather not change the password as it is quite long and complex and I just memorized it. Login works ...
6
votes
3answers
5k views

Why do some characters show as squares in Chrome?

For example in the dev tools I get something like: Some of these squares are at the end of lines, initially I thought they were carriage returns but it turns out they aren't. Also, squares appear ...
6
votes
3answers
681 views

How to replace all percent-encoded UTF-8 substrings with plain UTF-8 text?

I've got a html file with a lot of %-encoded UTF-8 text in URLs. For example "%D1%80%D0%B5%D1%81%D1%83%D1%80%D1%81%D1%8B" stands for "ресурсы" ("resources" in Russian). The task is to replace all ...
1
vote
1answer
168 views

Duplicity and GPG: charset conversion

I'm using duplicity on OS X Mavericks. If I run my duplicity command, I get the following error: ===== Begin GnuPG log ===== gpg: conversion from `US-ASCII' to `utf-8' failed: Illegal byte sequence ...
2
votes
2answers
575 views

view file containing ISO-8859 text, CRLF line terminators, and escape sequences

I have stumbled upon a strange text file in an archive of an old dos game downloaded from some abandonware site. The text file is named WHY_NOT.TXT. I assume the file is some sort of .NFO file from ...
4
votes
3answers
3k views

Set Gnome terminal's default charset to utf8

Currently, I opened my gnome terminal and the default charset is gb2312. How can set the default charset to utf8 in Gnome terminal? I checked the preference setting but seemed not to find it. ...
2
votes
1answer
55 views

Can't obtain the connected forms of some unicode grapheme clusters with combining diacritical marks

In Arabic and some other languages a character has two or three forms which will find its appropriate one automatically when you type a word (according to its previous and next characters). How can ...
2
votes
1answer
67 views

Mail character set for norwegian characters

When extracting a message from my inbox containing the norwegian letter "ø" I find that it is encoded with 3 bytes as "=F8" or hexadecimal "0x3d4638". In UTF-8 the letter "ø" is encoded with two bytes ...
5
votes
2answers
538 views

How to fix diacritics in PDF forms in Evince?

I happen to live in Czech Republic and we have such letters like ě, š, č, ť, etc in the alphabet and use them everywhere, including addresses. Today I was filling a PDF form in Evince and noticed that ...
-1
votes
2answers
208 views

How can I force sed to continue parsing when it hits an “invalid” character?

I have a big playlist in .pls format (created with deadbeef playing software) which also contains some badly-encoded strings. Whether this is a bug in deadbeef or one of its plugins does not really ...
3
votes
0answers
52 views

How to add EBCDIC support on debian/ubuntu

I have to add EBCSIC text support to deal with IBM OS text messages. I have succesfuly added EBCDIC support to a CentOS box by adding lang and encoding to /etc/i18n/sysconfig. I searched in the docs ...
1
vote
5answers
367 views

Deleting all files that begins with parenthesis

I have many files in a directory that begins with parenthesis. They are generated by Dropbox due to conflict. Any combination of escaping does not seem to help: rm -rf "(*" rm -rf "\(*" rm -rf \(* ...
5
votes
1answer
2k views

terminal: displaying special characters

In some programs, such as htop the lines and frames are not displayed correctly. Instead, they are displayed as - and /. But on another machine, they are displayed correctly as proper lines: I ...
5
votes
2answers
831 views

Lowercase all but the first (Uppercase) letter from UPPERCASE in cyrillic

To make all letters a lower case except the first letter. The first letter would look like "Uppercase" after I changed (from UPPERCASE in cyrillic). The rest (not UPPERCASED) leave unchanged. I'm ...
3
votes
4answers
1k views

Matlab gets “crazy” after type special characters

I'm using a computer running Ubuntu and I'm connected through ssh to a RedHat machine where I use Matlab in command line mode (matlab -nodesktop). Matlab version is 7.10.0.499 (R2010a). My problem ...
2
votes
0answers
59 views

how to guess and rename file from deleted rar has invalid encoding

i have many files and folders that extracted from many rar from jdownloader, and deleted after extract, some of it names are: �������[�ς݂��ς݂� - �Ȃ񂾂��ꂭ������ (invalid encoding) �R�e���} (invalid ...
2
votes
2answers
2k views

What are the implications of setting LC_ALL on Solaris

To support an application we just installed, I've been asked to add this command to the /etc/profile file on our Solaris 10 UNIX server: LC_ALL=en_US.UTF-8; export LC_ALL The effect of this changes ...
1
vote
2answers
2k views

How to specify attachment encoding in mutt command line?

I'm trying to send an email with attachment from a perl script. First, I create the attachment (an xml file): open(XMLFILE, ">:utf8", $xmlfile); print XMLFILE "<?xml version=\"1.0\" ...
3
votes
1answer
839 views

Encoding of a zip file

When I extract some file (mp3) with japanese or korean characters within its filename on Windows, they appear normally, but when I extract it on Linux (Ubuntu/Mint) they appear as something like this ...
5
votes
3answers
5k views

How to fix Russian letters in a Wine application when adjusting LANG does not help?

An application with a Russian UI shows question marks instead the most of its text (some elements are rendered ok though). The system locale is English (en_US, en_IE), I've tried Ubuntu ant XUbuntu ...
7
votes
3answers
10k views

How to specify characters using hexadecimal codes in `grep`?

I am using following command to grep character set range for hexadecimal code 0900 (instead of अ) to 097F (instead of व). How I can use hexadecimal code in place of अ and व? bzcat archive.bz2 | grep ...
3
votes
1answer
647 views

Vim powerline with broken encoding

I trying to learn using vim, I am enjoining a lot, but I copied the vimrc from github and I would like to understand what is causing the break of powerline encoding. . In the vimrc file I have: set ...
5
votes
3answers
5k views

Characters encodings supported by more, cat and less

I have a text file encoded as following according to file: ISO-8859 text, with CRLF line terminators This file contains French's text with accents. My shell is able to display accent and emacs ...
1
vote
0answers
382 views

Convert ASCII-8BIT to UTF-8 using OSX' .bash_profile

When running certain rails scripts in OSX' Terminal, I get warnings such as the following: unable to convert "\xA9" from ASCII-8BIT to UTF-8 for ...
1
vote
1answer
507 views

Polish chars in gedit (Fedora 18)

I locked my Fedora18 computer (set to Polish). Later I couldn't log in (password was accepted, but I didn't see my desktop). I used pkill -KILL -u tymon command in root mode (Ctrl+Alt+f2). After ...
2
votes
0answers
56 views

Search in evolution does not find umlaut in different encoding

I use Evolution 3.6.4 on Fedora 18 x86_64, my main language is German. I got a mail (on my imapx account) encoded in iso-8859-15 with the word "hügelig" in it. The mail is displayed correctly in ...
4
votes
2answers
205 views

Strange encoding in Slackware

I am a student and learning about operating systems, I was modifying a file using vi, when I accidentally did something (I don't remember what) that change the codification/encoding and now this is ...
7
votes
2answers
4k views

Effect of $LANG on terminal

I'm trying to learn how the $LANG variable behaves with gnome-terminal (and its character encoding preference option). I've been using iso8859-1 (latin1) as my main character-set and all my filenames ...
3
votes
1answer
319 views

Encoding Problem: Qt Apps delete all non ASCII chars from files

Qt applications are deleting non-latin characters from ISO-8859 encoded files on my Gentoo system. Actually I'm trying to merge two German files with KDiff3 and P4Merge (making "Whlen" out of ...
4
votes
2answers
652 views

What would break if the C locale was UTF-8 instead of ASCII?

The C locale is defined to use the ASCII charset and POSIX does not provide a way to use a charset without changing the locale as well. What would happen if the encoding of C were switched to UTF-8 ...
6
votes
2answers
816 views

Same file, different filename due to encoding problem?

I was about to diff a backup from it's source to manually verify that the data is correct. Some chars, like åäö, is not shown correctly on the original data but as the clients (over samba) interpret ...
10
votes
2answers
13k views

ssh and character encoding

When I ssh into my VPS, I have irssi running in screen. When someone sends a unicode character (such as © or €), irssi displays garbage when I use it via the screen in a ssh session. If I connect to ...
4
votes
4answers
4k views

How to convert hex chars to normal chars?

I tried, but I'm stuck at "escaping" the "sed's": sed -i 's/\\x0/NUL/g' $1 sed -i 's/\\x1/SOH/g' $1 sed -i 's/\\x2/STX/g' $1 sed -i 's/\\x3/ETX/g' $1 sed -i 's/\\x4/EOT/g' $1 sed -i 's/\\x5/ENQ/g' ...
4
votes
1answer
2k views

How can I change locale/encoding to avoid getting weird characters in terminal?

I have installed tree on my ubuntu box, when i connect to it from Putty and I launch tree i get this : $ tree âââ html.vim -> xml.vim âââ js.vim -> xml.vim âââ xml.vim instead of this : $ ...
17
votes
3answers
6k views

bulk rename (or correctly display) files with special characters

I have a bunch of directories and subdirectories that contain files with special characters, like this file: robbie@phil:~$ ls test�sktest.txt test?sktest.txt Find reveals an escape sequence: ...
4
votes
3answers
2k views

russian letters in gentoo console as '?'

I managed to get russian language working in my Gnome apps, but neither in the console (Alt-Shift-F2 and such) nor in the Gnome Terminal valyagentoo va1en0k # locale LANG=ru_RU.UTF-8 ...
0
votes
1answer
3k views

Script failing with “command not found: ^M”

When I try to run the following script in zsh, via the command /bin/zsh ~/.set_color_scheme.sh I get the following error: command not found: ^M The script has u+x permissions and it used to work on ...
8
votes
2answers
922 views

Why are wc -m and wc -c different?

As a C programmer, I was surprised to see that wc -c (which count the number of bytes), and wc -m (which counts the number of characters) output very different results for a long, text file of mine. I ...
2
votes
1answer
1k views

file, awk and iconv to automatically adapt tex-file encoding

I have a set of tex-files with mixed encodings, e.g. (subset of output of file -i *.tex) f1.tex: text/plain; charset=utf-8 f2.tex: text/plain; charset=utf-8 f3.tex: text/x-tex; charset=us-ascii ...
3
votes
1answer
488 views

tr not replacing apostrophe

I want to convert all apostrophes in this file to X: Bob's book Bob’s book Bob′s book # (Might look the same but actually different) The first apostrophe is replaced as expected: $ cat file | tr ...