I have many HTML files containing mixed unicode strings like \303\243 and printable characters like %s. What I'd like to do is converting the unicode strings into printable characters in a safe way.

I found that printf from GNU coreutils converts them automatically, but I also learned the hard way some time ago that printf is not trustworthy if you don't know what is going to be printed:

$ env printf "$(cat filename.htm)"
printf: %20M: invalid conversion specification

I also know that bash can do conversions like this, but I'd have to loop through the file with a regex, which is not safe at all:

$ echo $'\303\243'

Is there such a command-line tool that can process these files in a clean and safe way?

  • Do you know what encoding the files have? Is it in the <meta> tag of the HTML files, and is it correct? (It is if the html files show correctly in your browser). If the echo example gives the right 'printable' character this would be UTF-8. Any possibility of uploading one of the files somewhere or give a small snippet here that has header and a small part of the body of the HTML? – Anthon Apr 14 '13 at 4:55
  • So; is this correct? The file have literal \303\243 in the <body>, and you want to translate those escape sequences to unicode bytes? – Runium Apr 14 '13 at 4:59
  • Thanks to both of you for all the help! This seems to have been solved! Just for the record, the HTML files actually don't display properly in the web browser. I'm reconstructing pages sniffed with tshark. It'd be easier with Wireshark, but I'm doing it from the command line. – Teresa e Junior Apr 14 '13 at 15:56

If the files don't have other backslashes:

$ printf %b\\n 'aa\303\243'

If they do, you could double backslashes that aren't followed by integers:

$ printf %b\\n "$(sed -E 's/\\/\\\\/g;s/\\(\\[0-7])/\1/g' <<< '\\a\na\303\243')"
  • 1
    It seems printf becomes safe if a conversion specification is given: env printf "%b\n" "$(cat filename.htm)" Thank you! – Teresa e Junior Apr 14 '13 at 15:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.