Tagged Questions

The tag has no wiki summary.

learn more… | top users | synonyms

0
votes
0answers
20 views

Change encoding of a text file in-place: iconv alternative? [duplicate]

Apparently, iconv is unable to use the same file as input and output file. Other tools, e.g. sed, have an -i switch that allows "updating" a file in-place. (Under the hood, temporary files are ...
5
votes
1answer
34 views

Can not use `cut -c` (`--characters`) with UTF-8?

The command cut has an option -c to work on characters, instead of bytes with the option -b. But that does not seem to work, in en_US.UTF-8 locale: The second byte gives the second ASCII character ...
3
votes
1answer
31 views

How to remove Russian and Arabic strings from a text file

Problem: I have text file with approximately 1M lines, each line consists of multiple words Some lines contain Russian or Arabic words (lets call them "bad lines"), I want to remove only these bad ...
0
votes
2answers
31 views

Can CentOS be set to pure EBCDIC?

I need to directly manipulate files from a mainframe. Text would be encoded in EBCDIC. Data can contain binary and packed decimal fields. I know I can deal with the data. I am trying to avoid the ...
0
votes
0answers
32 views

A quick and dirty way of finding invalid characters in a text file by gedit

In vonbrand's comment to How do I scan for invalid characters on gedit? A quick and dirty way would be to edit the file (probably adding and deleting something will be enough to make gedit ...
1
vote
0answers
22 views

What determines the encoding of a text file when its content is copied from somewhere? [closed]

In emacs and in gedit, when copy some text from a webpage, pdf file, etc and save it as a text file, how is the encoding method of the resulting text file chosen by the editor? Does the editor choose ...
0
votes
1answer
26 views

Mixture of encodings

For a text which is a mixture of different encodings, is there no way to find all encodings? And convert them to a single encoding such as utf8? I copied from different sources possibly with ...
1
vote
2answers
48 views

Emacs shows octal escapes for some characters in a UTF-8 file

I copied some math formulas into some files under gedit, using the default settings of gedit from websites, such as math.stackexchange.com, wikipedia, ... When I open the files now under emacs, the ...
-1
votes
2answers
53 views

How does rsync encode filenames?

By using strace on the active rsync process, I can see output like: read(4, "9\0\0\7\1\10\0\1\10\0\1\10\0\1\10\0\1\10\0\1\10\0\1\10\0\1\10\0\1\10\0\1"..., 16384) = 61 write(3, ...
1
vote
1answer
29 views

What does “position” mean in iconv error message?

I get error when converting a large file: $ iconv -f GB2312 -t UTF-8 2001.txt -o 2001_u.txt iconv: illegal input sequence at position 245256667 What does the position mean in this error? I tried it ...
1
vote
1answer
180 views

Converting documents to text with UTF-8 encoding

unoconv uses LibreOffice to convert any file that LibreOffice can convert. Now I need export to text with the UTF-8 character encoding. I use the following command: unoconv -f txt -e ...
11
votes
4answers
211 views

support for utf-8 encoding with lpr

While trying to send a text file to the printer via lpr from xterm, the content was corrupted beyond recognition, the cause of which was ultimately traced to the encoding of the file. If I instead ...
4
votes
2answers
155 views

How do I rename files with strange characters

I have file with greek or cyrillic characters. It is not owned by me, but by the web server user (www). I cannot use the shell as the web server user (www) or as root, but I've used a script ...
5
votes
1answer
190 views

How to display Chinese characters correctly on remote Red-Hat machine?

I am using Ubuntu14.04 to connect to a remote host. Which its version is: Linux version 2.6.32-431.11.5.el6.yyyzzz.x86_64 (gcc version 4.4.7 20120313 (Red Hat 4.4.7-4) (GCC) ) #1 SMP Thu Jul 3 ...
2
votes
3answers
92 views

How to convert html entities to readable text?

I want html number entities like ę and want to convert it to real character. I have emails mostly from linkedin that look like this: chciałabym zapytać, czy ...
1
vote
1answer
56 views

replace control characters in Emacs?

This is a small text: What\222s with All the Parentheses? In Emacs \222 above is shown as a control character (in red). It is supposed to be ', but I don't understand why it is \222 instead, ...
1
vote
3answers
138 views

Debian thinks my file system is encoded as ISO-8859-1

This may not be the appropriate site for this question, because it's more a complaint than a question, but here goes. Today I updated my computer from Mint 15 to Debian 7.6. And after I'd done that, ...
0
votes
2answers
1k views

iconv illegal input sequence- why?

While trying to convert a text file into its ASCII equivalent, I get error message that iconv: illegal input sequence at position. Command I use is iconv -f UTF-8 -t ascii//TRANSLIT file The ...
0
votes
0answers
236 views

AddDefaultCharset ISO-8859-1 not work

I have upgrade Apache from 2.2 to 2.4 and I am having problems with character encoding. In my pages I have è=è ò=ò ì=ì etc... In my httpd.conf I have commented the default value and added ...
3
votes
2answers
477 views

tr complains of “Illegal byte sequence”

I'm brand new to UNIX and I am using Kirk McElhearn's "The Mac OS X Command Line" to teach myself some commands. I am attempting to use tr and grep so that I can search for text strings in a regular ...
5
votes
1answer
110 views

Greek characters in mutt's sidebar (mailboxes names sourced via offlineimap)

Mutt's sidebar, does not read properly Greek characters (mailboxes named by using Greek characters). Anyhow, there is no such problem inside both the index and the pager, where Greek ...
3
votes
1answer
317 views

Process a file that starts with a BOM (FF FE)

I received a .csv file with the FF FE BOM: $ head -n1 dotan.csv | hd 00000000 ff fe 41 00 64 00 20 00 67 00 72 00 6f 00 75 00 |..A.d. .g.r.o.u.| When using awk to parse it I'm getting a bunch of ...
2
votes
1answer
45 views

Non-standard logins

As part of a sort of esoteric "hack" I'm putting together, I've modified my /etc/passwd and /etc/shadow files to result in a user called ニエット ("Niet") with a shell pointing to a custom script I have ...
8
votes
2answers
840 views

What is the ^M character called?

TexPad is creating it. I know that it is under some deadkey. I just cannot remember it is name. The blue character: I just want to mass remove them from my document. How can you type it?
2
votes
2answers
75 views

How to fix the accents in xterm top bar?

I don't have any problem with accent (I'm French) inside xterm. However, like you can see on this screenshot, the top bar of xterm does have some accents issues: However my Debian unstable system ...
1
vote
1answer
72 views

How to export string about France to Microsoft excel file?

One string file Str.rc about French language is here: ID_STR_BRIGHTNESS;,"Luminosité" ID_STR_CHILE_EASTER_ISLAND;,"Île de Pâques" ID_STR_CURRENT_CH;,"Saisie chaîne" ID_STR_DETAILS;,"Détails" ...... ...
11
votes
1answer
291 views

Command to retrieve the list of characters in a given character class in the current locale

What could be way to retrieve a list of all the characters in a given character class (like blank, alpha, digit...) in the current locale. For instance, LC_ALL=en_GB.UTF-8 that-command blank ...
3
votes
2answers
198 views

cp: what quotation marks are used in error messages?

I'm currently fiddling around with a dying harddisk, and while trying to cp data from it, I get errors like cp: error reading ‘brokenFile’: Input/output error # comparison: backtick: `, apostrophe ' ...
4
votes
0answers
102 views

weird output when typing letter c in shell (non english input)

I had keyboard set to Georgian language input. Forgot to switch it back to English and typed letter c in shell. c = ც english c key is mapped to georgian letter ც - if this makes any difference. ...
0
votes
1answer
349 views

iconv cannot convert given characters

I just wanted to convert a txt file to UTF-8, since cat displays it correctly, but vi or gedit doesn't: $ cat test.txt ># >‹ | || ° ├── └── _ __ $ iconv -f WINDOWS-1253 -t UTF-8 test.txt ...
1
vote
2answers
63 views

Incorrect output from cp, rm, and so on

When I run, say, cp, I get output like the following: # cp -v Foo Bar âFooâ -> âBarâ What's up with the weird â characters? Why is the shell doing this? It looks like some kind of strange ...
2
votes
0answers
37 views

Character encoding of .netrc

Does anyone know which character encoding is used in .netrc? As a German user, I sometimes put umlauts in passwords. So my question is, are these treated as latin1 or may I assume that I am doing ...
0
votes
3answers
425 views

echo bytes to a file

I'm trying to connect my rasberry Pi to some display using the i2c bus. To get started I wanted to manually write stuff, bytes in particular to a file. How do you write specific bytes to a file? I ...
1
vote
1answer
29 views

Why does SLES 11 corrupts given characters?

If I open a notepad and copy this from it: aña, to a file opened with vi on the machine, I get this: aña My question: Why? What could the problem be? Logfiles on the server has the same ...
1
vote
0answers
136 views

Character encoding issue with my linux install?

I just installed Ubuntu 12.04.4 64 bits, all seemed to be ok, when I started to work on console I noticed: that I couldn't get some accented characters typed in to the console, for example, if I ...
1
vote
1answer
37 views

Why doesn't `restore` display my UTF-8 encoded filenames correctly in interactive mode?

When using the interactive mode of the restore utility to restore backups made with dump UTF-8-encoded filenames doesn't display correctly, see example below. The ??s should be ös ... $ sudo restore ...
3
votes
2answers
221 views

MP3 tags Cyrillic chars

I have some mp3 files with Cyrillic chars on file names and also on their tags. I'm using audacious to play them. See the file info below: Is it possible to change some encoding to show the ...
2
votes
3answers
67 views

Which terminal encodings are default on Linux, and which are most common?

I need to make a decision regarding whether a complicated commercial program that I work on should assume a particular terminal encoding for Linux, or instead read it from the terminal (and if so, ...
9
votes
3answers
2k views

identify files with non-ASCII or non-printable characters in file name

In a directory size 80GB with approx 700,000 files, there are some file names with non-English characters in the file name. Other than trawling through the file list laboriously is there: An easy ...
4
votes
1answer
104 views

login with xdm and umlaut in password

I want use xdm to login to my machine and start my session. My password contains an umlaut. I'd rather not change the password as it is quite long and complex and I just memorized it. Login works ...
1
vote
1answer
140 views

Duplicity and GPG: charset conversion

I'm using duplicity on OS X Mavericks. If I run my duplicity command, I get the following error: ===== Begin GnuPG log ===== gpg: conversion from `US-ASCII' to `utf-8' failed: Illegal byte sequence ...
1
vote
3answers
5k views

How to change encoding from Non-ISO extended-ASCII text, with CRLF line terminators to UTF-8?

I have a txt file : $ file -i x.txt x.txt: text/plain; charset=unknown-8bit $ file x.txt x.txt: Non-ISO extended-ASCII text, with CRLF line terminators And there are some characters that are ...
4
votes
2answers
477 views

Can vim display ASCII characters only, and treat other bytes as binary data?

I already know vim -b, however, depending on the locale used, it displays multi-byte characters (like UTF-8) as single letters. How can I ask vim to only display ASCII printable characters, and ...
5
votes
1answer
2k views

Convert a .docx to a .pdf with pandoc

I am trying to convert a .docx received by mail to a correct pdf by using pandoc (I am using GNU/Linux). I have an error concerning characters encoding : $ pandoc file.docx -o file.pdf pandoc: ...
2
votes
1answer
53 views

Can't obtain the connected forms of some unicode grapheme clusters with combining diacritical marks

In Arabic and some other languages a character has two or three forms which will find its appropriate one automatically when you type a word (according to its previous and next characters). How can ...
2
votes
1answer
64 views

Mail character set for norwegian characters

When extracting a message from my inbox containing the norwegian letter "ø" I find that it is encoded with 3 bytes as "=F8" or hexadecimal "0x3d4638". In UTF-8 the letter "ø" is encoded with two bytes ...
-1
votes
2answers
181 views

How can I force sed to continue parsing when it hits an “invalid” character?

I have a big playlist in .pls format (created with deadbeef playing software) which also contains some badly-encoded strings. Whether this is a bug in deadbeef or one of its plugins does not really ...
3
votes
0answers
49 views

How to add EBCDIC support on debian/ubuntu

I have to add EBCSIC text support to deal with IBM OS text messages. I have succesfuly added EBCDIC support to a CentOS box by adding lang and encoding to /etc/i18n/sysconfig. I searched in the docs ...
1
vote
5answers
295 views

Deleting all files that begins with parenthesis

I have many files in a directory that begins with parenthesis. They are generated by Dropbox due to conflict. Any combination of escaping does not seem to help: rm -rf "(*" rm -rf "\(*" rm -rf \(* ...
5
votes
1answer
1k views

terminal: displaying special characters

In some programs, such as htop the lines and frames are not displayed correctly. Instead, they are displayed as - and /. But on another machine, they are displayed correctly as proper lines: I ...