I want to fix broken characters in a file by converting windows-1252 to windows-1251 I want to do mapping that looks like:
sed 's/[ÉÖÓÊÅÍÃØÙÇÕÚÔÛÂÀÏÐÎËÄÆÝß×ÑÌÈÒÜÁÞ¸éöóêåíãøùçõúôûâàïðîëäæýÿ÷ñìèòüáþ]/[ЁЙЦУКЕНГШЩЗХЪФЫВАПРОЛДЖЭЯЧСМИТЬБёйцукенгшщзхъфывапролджэячсмитьбю]/g' input.py > input.py

I can replace characters one by one like:

sed 's/î/о/g'  input.py > output1.py
sed 's/è/и/g'  output1.py > input.py

But what is the syntax for character mapping to fix all characters in one run?


This is what tr is for. Assuming those really are the characters you want to change and this isn't an encoding issue, what you need is:

tr '¨ÉÖÓÊÅÍÃØÙÇÕÚÔÛÂÀÏÐÎËÄÆÝß×ÑÌÈÒÜÁÞ¸éöóêåíãøùçõúôûâàïðîëäæýÿ÷ñìèòüáþ' \
   'ЁЙЦУКЕНГШЩЗХЪФЫВАПРОЛДЖЭЯЧСМИТЬБЮёйцукенгшщзхъфывапролджэячсмитьбю' < input.py > output.py

However, I suspect what you want to do is to change the encoding of the file, not the characters. I can't test this without an example file, but perhaps this:

 iconv -f windows-1252 -t windows-1251 input.py > output.py
  • I've fixed the replacement rule a bit and now your tr method works exactly as expected. Thank you!
    – Stepan
    Apr 29 '20 at 11:18
  • @Stepan great, thanks for the edit. Are you sure it works though? On my tests, your new replacement gave some very strange results, including sanskrit characters. This could be because I was testing on a UTF-8 file, however.
    – terdon
    Apr 29 '20 at 12:14
  • Also, to answer the OP's question, y is the sed to command to transliterate characters. Apr 29 '20 at 14:13
  • @StéphaneChazelas oh, duh! I should have thought of that! Might be worth posting as an answer.
    – terdon
    Apr 29 '20 at 14:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.