Unix command to convert character encoding in a .csv file

Question

I need a Unix command to convert a .csv file that is in UNICODE format to ANSI format.

The file is imported from the Cognos environment and I am unable to make any changes with the format in cognos.

Answer 1

You can use iconv to convert between encodings

iconv -f utf-8 -t ascii oldfile > newfile

Answer 2

• I noticed mine having similar issue but was not able to find what type of encoding it was.

• Notepad++ tells in my case that it was ANSI so the above command dint help me.

• Now there are many types of ANSI so the best way to check that was to type

  iconv -l

• This lists all the encoding format names. So in my case I narrowed that to:

  *iconv -l|grep -i ansi
  ANSI_X3.4-1968//
  ANSI_X3.4-1986//
  ANSI_X3.4//
  ANSI_X3.110-1983//
  ANSI_X3.110//
  MS-ANSI//*
  

• Since I dint know the specific ANSI I tried almost all of them until I found it was MS-ANSI - I should have known, since I received the file from a Microsoft system. So then I converted using this command:

  iconv -f MS-ANSI -t utf-8 sample.csv > sampleUTF8.csv

• So in the same way list out the UNICODE types in iconv -l and reverse the order to get the ANSI type you desire and would be simple.