Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I have a txt file :

$ file -i x.txt
x.txt: text/plain; charset=unknown-8bit
$ file x.txt 
x.txt: Non-ISO extended-ASCII text, with CRLF line terminators

And there are some characters that are incorrectly encoded :

trwa³y, sta³y, usuwaæ

How can I change this file's encoding to UTF-8 ? I have tried the following way so far :

$ iconv -f ASCII -t UTF-8 x.txt
                puiconv: illegal input sequence at position 4

Maybe I should somehow use extended ASCII ( high ASCII ) but cannot find it in iconv's encoding list.

share|improve this question
1  
can you upload the file somewhere? –  janos Jan 7 at 19:44
    
There is a handy list of 8-bit ISO encodings, all shown side-by-side, here. Do any of them look close to what you observe in your file? For example, if you think "sta³y" should be "stacy", then find which encoding has a "c" for whatever the strange hex code is in that word. –  John1024 Jan 7 at 20:20

3 Answers 3

up vote 3 down vote accepted

file tells you “Non-ISO extended-ASCII text” because it detects that this is:

  • most likely a “text” file from the lack of control characters (byte values 0–31) other than line breaks;
  • “extended-ASCII” because there are characters outside the ASCII range (byte values ≥128);
  • “non-ISO” because there are characters in the 128–159 range (ISO 8859 reserves this range for control characters).

You have to figure out which encoding this file seems to be in. You can try Enca's automatic recognition. You might need to nudge it in the right direction by telling it in what language the text is.

enca x.txt
enca -L polish x.txt

To convert the file, pass the -x option: enca -L polish x.txt -x utf8 >x.utf8.txt

If you can't or don't want to use Enca, you can guess the encoding manually. A bit of looking around told me that this is Polish text and the words are trwały, stały, usuważ, so we're looking for a translation where ³ł and æż. This looks like latin-2 or latin-10 or more likely (given “non-ISO” CP1250 which you're viewing as latin1. To convert the file to UTF-8, you can use recode or iconv.

recode CP1250..utf8 <x.txt >x.utf8.txt
iconv -f CP1250 -t UTF-8 <x.txt >x.utf8.txt
share|improve this answer

Did you try to find out what exact encoding is x.txt? You'll get a list of supported encodings with

iconv --list

Sometimes it happens to me that I get a mismatch between latin1 and utf8. Then it often helps to convert it from and back to utf8 and vice versa.

share|improve this answer

Open the text file with gedit and in the "save as.." dialog you will see the current encoding.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.