Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I have many HTML files containing mixed unicode strings like \303\243 and printable characters like %s. What I'd like to do is converting the unicode strings into printable characters in a safe way.

I found that printf from GNU coreutils converts them automatically, but I also learned the hard way some time ago that printf is not trustworthy if you don't know what is going to be printed:

$ env printf "$(cat filename.htm)"
printf: %20M: invalid conversion specification

I also know that bash can do conversions like this, but I'd have to loop through the file with a regex, which is not safe at all:

$ echo $'\303\243'
ã

Is there such a command-line tool that can process these files in a clean and safe way?

share|improve this question
    
Do you know what encoding the files have? Is it in the <meta> tag of the HTML files, and is it correct? (It is if the html files show correctly in your browser). If the echo example gives the right 'printable' character this would be UTF-8. Any possibility of uploading one of the files somewhere or give a small snippet here that has header and a small part of the body of the HTML? –  Anthon Apr 14 '13 at 4:55
    
So; is this correct? The file have literal \303\243 in the <body>, and you want to translate those escape sequences to unicode bytes? –  Sukminder Apr 14 '13 at 4:59
    
Thanks to both of you for all the help! This seems to have been solved! Just for the record, the HTML files actually don't display properly in the web browser. I'm reconstructing pages sniffed with tshark. It'd be easier with Wireshark, but I'm doing it from the command line. –  Teresa e Junior Apr 14 '13 at 15:56

1 Answer 1

up vote 1 down vote accepted

If the files don't have other backslashes:

$ printf %b\\n 'aa\303\243'
aaã

If they do, you could double backslashes that aren't followed by integers:

$ printf %b\\n "$(sed -E 's/\\/\\\\/g;s/\\(\\[0-7])/\1/g' <<< '\\a\na\303\243')"
\\a\naã
share|improve this answer
    
It seems printf becomes safe if a conversion specification is given: env printf "%b\n" "$(cat filename.htm)" Thank you! –  Teresa e Junior Apr 14 '13 at 15:51

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.