How to remove duplicate lines inside specifc tag in a XML file

Question

Suppose you have the following file:

...
<tag2>
    a
    b
    c
    a
</tag2>
...
<tag2>
    x
    y
    y
    z
    x    
</tag2>

How to remove the duplicate lines inside each <tag2> like the example below?

...
<tag2>
    a
    b
    c
</tag2>
...
<tag2>
    x
    y   
    z
</tag2>

I would like to search every file in the directory and sub directories and removed these duplicates.

Answer 1

XSLT 2.0 solution:

<xsl:template match="tag2">
  <tag2>
    <xsl:value-of select="distinct-values(tokenize(., '&#xa;'))"/>
  </tag2>
</xsl:template>

Answer 2

Unsure how complex your file could be, but for the example given this appears to work.

$ awk '/^<[a-z]/{print;delete z}!/^</{z[$0]=1}/^<\//{for(x in z){print x}print}' file1
<tag2>
    a
    b
    c
</tag2>
<tag2>
    x
    y
    z
</tag2>
$

Commented version

awk '/^<[a-z]/ {         # If start tag
         print           #     Print line
         delete z        #     Clear array
     } !/^</ {           # If not a tag
         z[$0]=1         #     Store line
     } /^<\// {          # If end tag
         for(x in z) {   #     For each array entry
             print x     #         Print array entry
         }
         print           #     Print end tag
     }' file1