Using sed to extract text between 2 tags

Question

I have a .xml file and I'm trying to do a "groupinstall" on a RHEL6 machine as there are several hundred libraries in that .xml file... (close to 16 000 lines).

I'm therefore trying to extract the group names contained in the .xml file that has this structure:

<b>
<group>
<id> group name </id>
   <packages>
   ...
   </packages>
<id> group name 2 </id>
   <packages>
   ...
   </packages>
<id> etc... </id>
</group>
</b>

Basically, this is what I've tried:

sed -n '/<id>/,/<\/id>/p' test1.txt > test2.txt

I copied the .xml file to test1.txt. I'm trying to extract the group names from the test1.txt to a second file called test2.txt. However, with the line above, it is extracting everything from the FIRST <id> tag to the last </id> tag in my file. How can I change my code to extract it several times?

My second question would be: does the -downloadonly plugin work as well with groups for yum?

Answer 1

Sounds like what you need is more something along the lines of

sed -n 's:.*<id>\(.*\)</id>.*:\1:p'

(assuming like in your sample that the <id> and </id> are on the same line and that there's only one <id>...</id> per line).

Or use a XML-aware tool:

xmlstarlet sel -t -v '//id' -n

Answer 2

$ echo '<id>I am a sample group</id>' | sed 's/<\/\?[^>]\+>//g'
I am a sample group
$

This will work with any tag, of course also with <a href="...">...</a> anchors. No GNUisms used — basic regex support in sed will suffice.
However: please note that both opening and closing tags must be in the same line, otherwise the statement would have to be rewritten again.

Answer 3

Please give a try with

xml_grep 'id' file.xml --text_only

Answer 4

This is XML, you should use an XML parser. Here's a solution using XMLStarlet:

$ xml sel -t -v '//group/id' -nl data.xml
 group name
 group name 2

The XPath expression //group/id will select any id node beneath a group node. The -t -v means "use the following template to extract values". The -nl at the end will make sure that the output is ended with a newline.

The example above uses an XML file which is identical to yours, but with any line containing ... removed.

Answer 5

I read this post looking to solve the issue of extracting Reqd. Packages from the RHEL 7.3 DVD repos.xml, which I think is exactly what the author above was trying to do. So I hope this script may help someone else… I have used it many times now.

So I needed to install the "GNOME DESKTOP" group to my "Minimal Install" RHEL7 server that had no X/GUI configured.

[root@rac01]# yum group list
Loaded plugins: ulninfo
There is no installed groups file.

Hmmmmm… no group list on the DVD for yum (yep, I tried all the usual "google" fix-its and never worked) so resorted to hard source the list from xml.

1. Mount the DVD.
2. Find the XML file with my required package list.
3. Extract the list of group of packages.
4. Loop through the packages list and install (incl. dependencies).

5. Assuming you have run createrepo /your/local_rpms/dir.

   sudo su -
   mkdir /mnt/sr0
   mount /dev/sr0 /mnt/sr0
   cd /mnt/sr0
   
   FILE=$(find . -name "*.xml" | xargs grep '<id>gnome-desktop<\/id>'| cut -d: -f1)
   PKGLIST=$(sed -n '/<id>gnome-desktop<\/id>/,/<\/packagelist>/p' $FILE \
   | sed  -n  '/^ *<packagelist> *$/,/^ *<\/packagelist> *$/{/<packagereq type>/{d};p}' \
   | cut -d'>' -f2 \
   | cut -d'<' -f1)
   
   for p in ${PKGLIST}
      do
       yum deplist ${p}* | awk '/provider:/ {print $2}' | sort -u | xargs yum -y install
   done