I have a number of XML files containing sanskrit texts to be converted to tex. Latex has a maximum of 63 characters per word for its hyphenation to work, everything longer than that will not be hyphenated. Now I would like to grep my files for these words, only that grep doesn't appear to be the right tool here. Some of the words use IAST encoding, others Devanāgarī. I suppose a perl one-liner could do that?
1 Answer
In an attempt to give this Q a proper answer, based - on - the - comments (heeding Sobrique's note that parsing XML should really be done with an XML parser):
perl -CSD -lne 'print for /\w{63,}/g' input-file-here
answered Jun 6, 2016 at 16:36
-
Actually I had that time already found the solution. The files being XML files didn't really matter. And
\w(after switching input and output to utf-8) is better than\S, as otherwise punctuation and parts of the markup are matched, too.– muk.liCommented Jun 6, 2016 at 17:31 -
I've updated it to use \w instead of \S -- I'm always a fan of what works!– Jeff Schaller ♦Commented Jun 6, 2016 at 17:48
perl -lne 'print for /\w{66,}/g'(untested)\wto work I would need a sanskrit locale? Like this it doesn't catch letters with diacritics, such as ī, and it doesn't handle Devanāgarī script at all.\S{63,}-- that will include punctuation, but should catch the long words too.-CDto read the input as utf8.-CSDto turn stdout to utf8, too.