5

I have a number of XML files containing sanskrit texts to be converted to tex. Latex has a maximum of 63 characters per word for its hyphenation to work, everything longer than that will not be hyphenated. Now I would like to grep my files for these words, only that grep doesn't appear to be the right tool here. Some of the words use IAST encoding, others Devanāgarī. I suppose a perl one-liner could do that?

8
  • 3
    perl -lne 'print for /\w{66,}/g' (untested) Mar 15, 2016 at 12:35
  • @StéphaneChazelas That's going in the right direction, but it seems for \w to work I would need a sanskrit locale? Like this it doesn't catch letters with diacritics, such as ī, and it doesn't handle Devanāgarī script at all.
    – muk.li
    Mar 15, 2016 at 12:51
  • 3
    How about \S{63,} -- that will include punctuation, but should catch the long words too. Mar 15, 2016 at 12:53
  • 1
    Just add -CD to read the input as utf8.
    – choroba
    Mar 15, 2016 at 12:55
  • 2
    @muk.li: Ah, so use -CSD to turn stdout to utf8, too.
    – choroba
    Mar 15, 2016 at 13:05

1 Answer 1

1

In an attempt to give this Q a proper answer, based - on - the - comments (heeding Sobrique's note that parsing XML should really be done with an XML parser):

perl -CSD -lne 'print for /\w{63,}/g' input-file-here
2
  • Actually I had that time already found the solution. The files being XML files didn't really matter. And \w (after switching input and output to utf-8) is better than \S, as otherwise punctuation and parts of the markup are matched, too.
    – muk.li
    Jun 6, 2016 at 17:31
  • I've updated it to use \w instead of \S -- I'm always a fan of what works!
    – Jeff Schaller
    Jun 6, 2016 at 17:48

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .