The tag has no wiki summary.

learn more… | top users | synonyms

0
votes
1answer
12 views

iconv terminated by signal 7

I was running a little bash script on some text files: find . -name "*.dat" -exec iconv --from-code='UTF-8' --to-code='ASCII//TRANSLIT' {} --output={} \; My machine is a Ubuntu 14.04 LTS. After a ...
2
votes
0answers
31 views

iconv cannot replace Ø

It appears that iconv cannot, for example, replace the letter Ø. It was also noted in the second answer to http://stackoverflow.com/questions/3371697/replacing-accented-characters-php I have two ...
3
votes
2answers
63 views

Why can't I grep this command line option?

The output of man icc contains -Ldir Tells the linker to search for libraries... Doing man icc | grep "\-L" gives back the "-Ldir" line while man icc | grep "\-Ldir" does not. What am ...
19
votes
5answers
10k views

Filtering invalid utf8

I have a text file in an unknown or mixed encoding. I want to see the lines that contain a byte sequence that is not valid UTF-8 (by piping the text file into some program). Equivalently, I want to ...
2
votes
2answers
58 views

Check files for conventions

I need a script that checks and lists all text files for project conventions. With conventions I mean for example: UTF-8 encoding No trailing white spaces Newline at the end of file No not-ascii ...
1
vote
1answer
61 views

Converting a UTF-8 file to ASCII (best-effort)

I have a file in UTF-8 that contains texts in multiple languages. A lot of it are people's names. I need to convert it to ASCII and I need the result to look as decent as possible. There are many ...
2
votes
0answers
65 views

$Unix character set conversion

I'm confused by character-sets in Unix. I have a CSV file downloaded via SFTP: $ file -ib myfile text/plain; charset=us-ascii The purpose for this character-set quest is that the data within file ...
2
votes
1answer
40 views

Change Text File Encoding without knowning the source encoding

I want to change the charset encoding for a file in unix with a single command but since this will be an automated process it's impossible for me to know the source encoding. So I want a command that ...
4
votes
1answer
586 views

File name encoding wrong in KDE, but correct in GTK/tty1/

I'm on a new Linux (installed yesterday). When I access files from old partitions their file names have incorrect encoding (ä instead of ä, same for other umlauts) inside KDE apps like dolphin and ...
1
vote
1answer
81 views

How to export string about France to Microsoft excel file?

One string file Str.rc about French language is here: ID_STR_BRIGHTNESS;,"Luminosité" ID_STR_CHILE_EASTER_ISLAND;,"Île de Pâques" ID_STR_CURRENT_CH;,"Saisie chaîne" ID_STR_DETAILS;,"Détails" ...... ...
0
votes
0answers
23 views

jumbled character set

In my Linux env, for some unknown! reason when I process a raw unformatted file which I copied from Win7prof (was encoded in utf-16), the resulting file is all jumbled. It will print OK in Linux ...
5
votes
1answer
99 views

Some apps doesn't accept some characters from the «Compose» key

The problem is that the compose key works fine, but some application doesn't accept some characters from it. E.g. I can paste the character ∞ in Emacs (Compose+8+8), but this won't work in FireFox, ...
1
vote
1answer
54 views

File weird character encoding [closed]

I'm on ubuntu 13.10 and am writing a program in eclipse that's supposed to output my result in a file. I'm building an LZW compression and decompression program that uses character conversion to its ...
1
vote
0answers
36 views

Why doesn't this sed substitution for a non-ASCII byte work?

I tried this substitution with GNU sed on OS X (4.2.2 installed through Homebrew). But it doesn't work. printf "Hello\x92 World" | gsed -r "s/[\x92]/'/g" The expected output is: Hello' World The ...
0
votes
0answers
22 views

Change encoding of a text file in-place: iconv alternative? [duplicate]

Apparently, iconv is unable to use the same file as input and output file. Other tools, e.g. sed, have an -i switch that allows "updating" a file in-place. (Under the hood, temporary files are ...
5
votes
1answer
71 views

Can not use `cut -c` (`--characters`) with UTF-8?

The command cut has an option -c to work on characters, instead of bytes with the option -b. But that does not seem to work, in en_US.UTF-8 locale: The second byte gives the second ASCII character ...
3
votes
1answer
70 views

How to remove Russian and Arabic strings from a text file

Problem: I have text file with approximately 1M lines, each line consists of multiple words Some lines contain Russian or Arabic words (lets call them "bad lines"), I want to remove only these bad ...
0
votes
2answers
50 views

Can CentOS be set to pure EBCDIC?

I need to directly manipulate files from a mainframe. Text would be encoded in EBCDIC. Data can contain binary and packed decimal fields. I know I can deal with the data. I am trying to avoid the ...
2
votes
1answer
840 views

How to get rid of special characters in the output of make [closed]

When I run gcc, it outputs errors but the function names are surrounded by special characters like this ‘setup_connection’: This same is repeated with variable names. I am wondering if the ...
3
votes
2answers
7k views

Bash/Korn shell script edited on Windows throws error '…^M: not found'

Some background on the issue: I am new to scripting and wrote a simple Bash script in Windows using Notepad++. cxStopAllServicesOnSERVER.sh #!/usr/bin/bash cd "/some/path" echo "Hi. Logs can be ...
4
votes
2answers
727 views

Can vim display ASCII characters only, and treat other bytes as binary data?

I already know vim -b, however, depending on the locale used, it displays multi-byte characters (like UTF-8) as single letters. How can I ask vim to only display ASCII printable characters, and ...
0
votes
0answers
46 views

A quick and dirty way of finding invalid characters in a text file by gedit

In vonbrand's comment to How do I scan for invalid characters on gedit? A quick and dirty way would be to edit the file (probably adding and deleting something will be enough to make gedit ...
1
vote
0answers
26 views

What determines the encoding of a text file when its content is copied from somewhere? [closed]

In emacs and in gedit, when copy some text from a webpage, pdf file, etc and save it as a text file, how is the encoding method of the resulting text file chosen by the editor? Does the editor choose ...
1
vote
2answers
59 views

Emacs shows octal escapes for some characters in a UTF-8 file

I copied some math formulas into some files under gedit, using the default settings of gedit from websites, such as math.stackexchange.com, wikipedia, ... When I open the files now under emacs, the ...
0
votes
1answer
26 views

Mixture of encodings

For a text which is a mixture of different encodings, is there no way to find all encodings? And convert them to a single encoding such as utf8? I copied from different sources possibly with ...
0
votes
2answers
77 views

How does rsync encode filenames?

By using strace on the active rsync process, I can see output like: read(4, "9\0\0\7\1\10\0\1\10\0\1\10\0\1\10\0\1\10\0\1\10\0\1\10\0\1\10\0\1\10\0\1"..., 16384) = 61 write(3, ...
1
vote
1answer
39 views

What does “position” mean in iconv error message?

I get error when converting a large file: $ iconv -f GB2312 -t UTF-8 2001.txt -o 2001_u.txt iconv: illegal input sequence at position 245256667 What does the position mean in this error? I tried it ...
2
votes
3answers
183 views

How to convert html entities to readable text?

I want html number entities like ę and want to convert it to real character. I have emails mostly from linkedin that look like this: chciałabym zapytać, czy ...
11
votes
4answers
287 views

support for utf-8 encoding with lpr

While trying to send a text file to the printer via lpr from xterm, the content was corrupted beyond recognition, the cause of which was ultimately traced to the encoding of the file. If I instead ...
1
vote
1answer
440 views

Converting documents to text with UTF-8 encoding

unoconv uses LibreOffice to convert any file that LibreOffice can convert. Now I need export to text with the UTF-8 character encoding. I use the following command: unoconv -f txt -e ...
0
votes
1answer
170 views

Change default Encoding in Terminator

The Title stays pretty much for itself. Tried to figure it out myself but I can´t find a config-setting nor a option in the Preferences for the Encoding. Does anybody know how to change the default ...
4
votes
2answers
355 views

How do I rename files with strange characters

I have file with greek or cyrillic characters. It is not owned by me, but by the web server user (www). I cannot use the shell as the web server user (www) or as root, but I've used a script ...
5
votes
1answer
528 views

How to display Chinese characters correctly on remote Red-Hat machine?

I am using Ubuntu14.04 to connect to a remote host. Which its version is: Linux version 2.6.32-431.11.5.el6.yyyzzz.x86_64 (gcc version 4.4.7 20120313 (Red Hat 4.4.7-4) (GCC) ) #1 SMP Thu Jul 3 ...
1
vote
1answer
69 views

replace control characters in Emacs?

This is a small text: What\222s with All the Parentheses? In Emacs \222 above is shown as a control character (in red). It is supposed to be ', but I don't understand why it is \222 instead, ...
1
vote
3answers
248 views

Debian thinks my file system is encoded as ISO-8859-1

This may not be the appropriate site for this question, because it's more a complaint than a question, but here goes. Today I updated my computer from Mint 15 to Debian 7.6. And after I'd done that, ...
5
votes
1answer
2k views

Convert a .docx to a .pdf with pandoc

I am trying to convert a .docx received by mail to a correct pdf by using pandoc (I am using GNU/Linux). I have an error concerning characters encoding : $ pandoc file.docx -o file.pdf pandoc: ...
1
vote
2answers
2k views

iconv illegal input sequence- why?

While trying to convert a text file into its ASCII equivalent, I get error message that iconv: illegal input sequence at position. Command I use is iconv -f UTF-8 -t ascii//TRANSLIT file The ...
0
votes
0answers
493 views

AddDefaultCharset ISO-8859-1 not work

I have upgrade Apache from 2.2 to 2.4 and I am having problems with character encoding. In my pages I have è=è ò=ò ì=ì etc... In my httpd.conf I have commented the default value and added ...
4
votes
2answers
949 views

tr complains of “Illegal byte sequence”

I'm brand new to UNIX and I am using Kirk McElhearn's "The Mac OS X Command Line" to teach myself some commands. I am attempting to use tr and grep so that I can search for text strings in a regular ...
5
votes
1answer
147 views

Greek characters in mutt's sidebar (mailboxes names sourced via offlineimap)

Mutt's sidebar, does not read properly Greek characters (mailboxes named by using Greek characters). Anyhow, there is no such problem inside both the index and the pager, where Greek ...
3
votes
1answer
518 views

Process a file that starts with a BOM (FF FE)

I received a .csv file with the FF FE BOM: $ head -n1 dotan.csv | hd 00000000 ff fe 41 00 64 00 20 00 67 00 72 00 6f 00 75 00 |..A.d. .g.r.o.u.| When using awk to parse it I'm getting a bunch of ...
2
votes
1answer
45 views

Non-standard logins

As part of a sort of esoteric "hack" I'm putting together, I've modified my /etc/passwd and /etc/shadow files to result in a user called ニエット ("Niet") with a shell pointing to a custom script I have ...
8
votes
2answers
1k views

What is the ^M character called?

TexPad is creating it. I know that it is under some deadkey. I just cannot remember it is name. The blue character: I just want to mass remove them from my document. How can you type it?
2
votes
2answers
90 views

How to fix the accents in xterm top bar?

I don't have any problem with accent (I'm French) inside xterm. However, like you can see on this screenshot, the top bar of xterm does have some accents issues: However my Debian unstable system ...
11
votes
1answer
313 views

Command to retrieve the list of characters in a given character class in the current locale

What could be way to retrieve a list of all the characters in a given character class (like blank, alpha, digit...) in the current locale. For instance, LC_ALL=en_GB.UTF-8 that-command blank ...
2
votes
3answers
773 views

sed, grep, or tr command that returns only Latin characters from a UTF-8 file

I am working with the text of the 300 tang poems, which unfortunately is a single file containing both Chinese and English. Since I am interested in 'extracting' the English, I hope to use sed, ...
7
votes
4answers
4k views

How to print all printable ASCII chars in CLI?

How can I list all the printable ASCII characters in the terminal?
3
votes
2answers
205 views

cp: what quotation marks are used in error messages?

I'm currently fiddling around with a dying harddisk, and while trying to cp data from it, I get errors like cp: error reading ‘brokenFile’: Input/output error # comparison: backtick: `, apostrophe ' ...
4
votes
0answers
112 views

weird output when typing letter c in shell (non english input)

I had keyboard set to Georgian language input. Forgot to switch it back to English and typed letter c in shell. c = ც english c key is mapped to georgian letter ც - if this makes any difference. ...
0
votes
1answer
564 views

iconv cannot convert given characters

I just wanted to convert a txt file to UTF-8, since cat displays it correctly, but vi or gedit doesn't: $ cat test.txt ># >‹ | || ° ├── └── _ __ $ iconv -f WINDOWS-1253 -t UTF-8 test.txt ...