7

I want html number entities like ę and want to convert it to real character. I have emails mostly from linkedin that look like this:

chciałabym zapytać, czy rozważa Pan takze udział w nowych projektach w Warszawie ? Obecnie poszukujemy specjalisty javascript/architekta z bardzo dobrą znajomością Angular.js do projektu, który dotyczy systemu, służącego do monitorowania i zarządzania flotą pojazdów. Zespół, do którego poszukujemy

I'm using clawsmail, switching to html don't convert it to text, I've try to copy and use

xclip -o -sel clip | html2text | less

but it didn't convert the entities. Is there a way to have that text using command line tools?

The only way I can think of is to use data:text/html,<PASTE THE EMAIL> and open it in a browser, but would prefer the command line.

18

With GNU recode:

recode html < file

If you don't have recode or HTML::Entities and only need to decode &#x<hex>; entities, you could do it by hand with:

perl -Mopen=locale -pe 's/&#x([\da-f]+);/chr hex $1/gie'
  • this work perfect c-v | html2text | recode html – jcubic Aug 8 '14 at 15:02
4

From How can I decode HTML entities? on StackOverflow, you may be able to implement a simple perl solution such as

perl -Mopen=locale -MHTML::Entities -pe '$_ = decode_entities($_)' email.txt

e.g. using your example text

$ perl -Mopen=locale -MHTML::Entities -pe '$_ = decode_entities($_)' email.txt
chciałabym zapytać, czy rozważa Pan takze udział w nowych projektach w Warszawie ? Obecnie poszukujemy specjalisty javascript/architekta z bardzo dobrą znajomością Angular.js do projektu, który dotyczy systemu, służącego do monitorowania i zarządzania flotą pojazdów. Zespół, do którego poszukujemy

With -Mopen=locale, I/O is done in the locale's character set. That includes input from email.txt. It looks like email.txt contains only ASCII characters (the whole point of encoding those characters using the &#x<hex>; notation I suppose), but if not you may need to adapt the above to also decode that file using the right charset (if it's not the same as the locale's one) instead of using open=locale.

  • 2
    You should use the -Mopen=locale option so that the text is output in the user's charset (and make that warning go away). – Stéphane Chazelas Aug 8 '14 at 15:01
2

A python 3.2+ version, can be used in a pipe:

python3 -c 'import html, sys; [print(html.unescape(l), end="") for l in sys.stdin]' < file
  • Cleaner: python3 -c'import html,sys;print(html.unescape(sys.stdin.read()), end="")' – ariddell Mar 27 '18 at 12:55
  • @ariddell : your version isn't line-by-line, and I wanted to preserve line boundaries; otherwise it blocks a pipe until everything is read on stdin (pipe is exhausted). – Aissen Mar 27 '18 at 15:09
-1

echo -e "\x01\x19" should do the trick.

  • to get up votes you should probably write shell code that will convert &#x119; to echo -e "\x01\x19" should be possible with sed. – jcubic Aug 8 '14 at 14:43
  • Also this don't work because it's one character and I don't get it when I run your command. – jcubic Aug 8 '14 at 14:45
  • \u119 work, but I'm not able to make it work with sed. So far I have c-v | sed -e 's/&#x\([^;]*\);/\\u\1/g' -e 's/.*/echo -e "&"/' | bash – jcubic Aug 8 '14 at 14:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.