I want to compare two UTF-8 encoded text file. Can Linux command diff and comm handle these encoding?

  • Note it is not Linux that does it. The kernel does not get involved with these matters. It is the Gnu tools that do it (If you are running a Linux based Unix system, then it will be Gnu). – ctrl-alt-delor Jun 22 '17 at 17:15

Why not?

2 text files in Russian

$ file -i test1.txt test2.txt                                                                                                                                                    
test1.txt: text/plain; charset=utf-8                                                                                                                                                                               
test2.txt: text/plain; charset=utf-8

$ cat test1.txt                                                                                                                                                                  

$ cat test2.txt                                                                                                                                                                  
Добрый день                                                                                                                                                                                                        

$ diff test1.txt test2.txt                                                                                                                                                       
< Привет                                                                                                                                                                                                           
> Добрый день                                                                                                                                                                                                      
  • How was this resolved? – Dejan Dec 30 '20 at 3:11

Use the -i parameter to force file to print information about the encoding

I created two files containing some caracters

One encoded in utf-8 and one encoded in iso-8859-1

$ file -i *
file1: text/plain; charset=utf-8
file2: text/plain; charset=iso-8859-1
  • I think they were asking about the diff and comm commands, not file – Jeff Schaller Feb 4 '17 at 11:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.