1

I have a utf-8 file containing some Turkish text inside. (My system is MacOSX)

$ file -I foo.merge
$foo.merge: text/plain; charset=utf-8

When I try to see some Turkish specific characters by using grep, there is no problem:

$ grep 'Emiroğlu' foo.merge
EMİROĞLU    Emiroğlu+Noun+A3sg+Pnon+Nom Emiroğlu+Noun+Prop+Noun+A3sg+P3sg+Nom   Emiroğlu+Noun+Prop+Noun+A3sg+Pnon+Nom   NOTFOUND

I can also see the file by using less command without any problem.

However if I try to do the following, the Turkish characters are not seen properly:

$ grep 'Emir' foo.merge | less
EMİROĞLU        ESC[1;35;40mESC[KEmirESC[mESC[Koğlu+Noun+A3sg+Pnon+Nom  ESC[1;35;40mESC[KEmirESC[mESC[Koğlu+Noun+Prop+Noun+A3sg+P3sg+Nom        ESC[1;35;40mESC[KEmirESC[mESC[Koğlu+Noun+Prop+Noun+A3sg+Pnon+Nom        NOTFOUND

Or the following also doesn't work:

$grep 'Emir' foo.merge > foo2.out
$less foo2.out

What could be the problem? Here is some additional information:

$ locale
LANG="en_US.utf-8"
LC_COLLATE="en_US.utf-8"
LC_CTYPE="en_US.utf-8"
LC_MESSAGES="en_US.utf-8"
LC_MONETARY="en_US.utf-8"
LC_NUMERIC="en_US.utf-8"
LC_TIME="en_US.utf-8"
LC_ALL="en_US.utf-8"
  • I guess (s)he meant the Turkish characters are not appeared in the second example as he seeks and this is a problem @Kusalananda – zwlayer Feb 3 at 10:43
  • @zwlayer Oh, I saw EMİROĞLU and thought that was the expected output. – Kusalananda Feb 3 at 10:46
  • @Kusalananda Yup, EMİROĞLU is the part of the expected output. But I guess problem occurs in the second part of that output where (s)he gets ESC[1;35;40mESC[KEmirESC[mESC[Koğlu instead of Emiroğlu – zwlayer Feb 3 at 10:50
  • 1
    Your grep is using terminal control sequences to highlight (color) the matched string, which less does not understand by default. Look for a grep option to turn off coloring (in GNU --color=never or --color=auto) or use less -R to tell it to understand terminal coloring. If you (want to) use these a lot you may want to make them aliases or functions in your shell profile. – dave_thompson_085 Feb 3 at 11:19
1

The Turkish characters look fine. However grep has inserted colour codes into the output.

Choices:

  • add option --color=never to grep (to remove colour).
  • add option -R to less (to tell less to interpret ASCII colour codes).

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.