3

I have a set of tex-files with mixed encodings, e.g. (subset of output of file -i *.tex)

f1.tex: text/plain; charset=utf-8
f2.tex: text/plain; charset=utf-8
f3.tex: text/x-tex; charset=us-ascii
f4.tex: text/plain; charset=iso-8859-1
f5.tex: text/plain; charset=us-ascii

I want to convert them all to utf-8, especially those encoded iso-8859-1. I can do this manually using (or similar)

iconv -f ISO-8859-1 -t UTF-8 f4.tex > tmp && mv tmp f4.tex

but I thought this would be nicely possible using a combination of awk with the above, i.e. running file -i parsing this for file/encoding combinations using awk and performing the encoding conversion accordingly using iconv.

My knowledge of awk is rather limited. I got no further than this:

$ file -i *.tex | awk '{print $1, $3}'
f1.tex: charset=utf-8
f2.tex: charset=utf-8
f3.tex: charset=us-ascii
f4.tex: charset=iso-8859-1
f5.tex: charset=us-ascii

Any help appreciated! Especially, I don't know how I can strip the colon : and the charset= substrings off the columns.

1 Answer 1

2

Seems it's much better to use sed here insted of awk:

file -i *tex | sed \ 
's/^\([^:]*\): .*set=\(.*\)/iconv -f \2 -t UTF-8 \1 > tmp \&\& mv tmp \1/e'

It would execute commands that sed will make based on file -i output. If you want to look at commands list without executing just remove e flag at the end of sed script like this:

file -i *tex | sed \
's/^\([^:]*\): .*set=\(.*\)/iconv -f \2 -t UTF-8 \1 > tmp \&\& mv tmp \1/'
2
  • I'm intrigued by the e flag, but it probably is a GNU extension? (That is, FreeBSD's sed doesn't know it.)
    – sr_
    Commented Apr 18, 2012 at 6:57
  • Yes, it is a GNU extension. I forgot to write about it. To use smth like that in non-GNU environment you can use contruction echo "ehco Hello" | sed 's/hc/ch/' | bash .
    – rush
    Commented Apr 18, 2012 at 8:00

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .