Looking for duplicate instances of a tag in a file

Question

Mutliple snippets of code exist in a file similar to the following:

<blah>Spread the peanut butter <ramout assot="f0123_fun10" bapel="2 or 6"/> on good looking bread <ramout assot="f0123_fun10" bapel="3 or 5"/> that does not have peanut butter <ramout assot="f0123_fun10" bapel="2 or 6"/> already on the bread this that and the other <ramout assot="f0123_fun10" bapel="4"/> with something else.</blah>

I am trying to find duplicate instances of the ramout tag in a single file. If the following exists:

<ramout assot="f0123_fun10" bapel="2 or 6"/> 

I want to know if it is repeated again within the opening and closing blah tags.

I've tried multiple things but one of the latest was the following:

grep -Eoi '<blah>.*([[:space:]]<ramout assot).*\1.*</blah>' *.xml | less

which returned nothing.

I also tried:

 grep -Eio '<blah>.*([[:space:]]<ramout assot="[a-z][0-9]{5}_fig[0-9]+" bapel="[0-9]+.*)' *.xml

which does not include the backreference but it also does not show all results. It looks like this is only showing the results that are one one line (do not span across a more than one line).

Should I use sed if I want to search for something that may or may not be on one line?

Is awk a viable candidate? I saw and tried: awk '/Start pattern/,/End pattern/' filename which returned more results but I am still not getting all results.

Any help being able to find a) all results in the entire file and separately b) all results that are duplicates within blah tags would be appreciated.

Expected results would look something like:

results for search a) showing all ramout results:

<ramout assot="f0123_fun10" bapel="2 or 6"/>
<ramout assot="f0123_fun10" bapel="3 or 5"/>
<ramout assot="f0123_fun10" bapel="2 or 6"/>
<ramout assot="f0123_fun10" bapel="4"/>

results for search b) showing duplicate results would show:

<ramout assot="f0123_fun10" bapel="2 or 6"/>

Answer 1

Using XMLStarlet (sometimes installed as xmlstarlet instead of just xml) to extract the relevant tags, then sort and uniq to find the duplicates:

$ xml sel -t -m '/blah/ramout' -c '.' -nl test.xml | sort | uniq -d
<ramout assot="f0123_fun10" bapel="2 or 6"/>

The xml command will match all <ramout> tags directly under the <blah> tag, and for each of these copy the tag followed by a newline to standard output.

sort sorts and uniq -d will extract any duplicate entries from the output of sort.

Answer 2

Something like this works ok in my tests:

awk -F"/>" -v RS="<ramout assot=" 'NR>1{print RS $1 FS}' file1

echo "Finding Cuplicates:"
awk -F"/>" -v RS="<ramout assot=" 'NR==1{next}seen[$1]++==1{print RS $1 FS}' file1

<ramout assot="f0123_fun10" bapel="2 or 6"/> 
<ramout assot="f0123_fun10" bapel="3 or 5"/> 
<ramout assot="f0123_fun10" bapel="2 or 6"/>
<ramout assot="f0123_fun10" bapel="4"/> 
Finding Cuplicates:              
<ramout assot="f0123_fun10" bapel="2 or 6"/>  

Test it online here

We make advantage of awk capabilitie to declare a custom record separator (RS) and custom field separator (FS). Above two commands can be combined in one awk offourse, this was just a test.