How to grep an xml block in an xml file using a keyword in ksh

Question

I have a file Sample.xml which contains a lot of services inside it and the structure looks like this

Notes:

1. I cannot use any XML parser tool as I don't have permission, read only

2. My xmllint version does not support xpath, and I cannot update it, read only

3. I don't have xmlstarlet and cannot install it

PROBLEM: INPUT: QUEUE NAME

OUTPUT: SERVICE BLOCK

sample INPUT: ABC.getme2

OUTPUT NEEDED:

<service name="GETME2" min="1" max="10" idleTime="300" backend="ABC">
                            <handlerContainer className="com.abc.xyz.wqere.abcqwere">
                            <handler className="com.abc.xyz.qweqweqwe.werwerwerwer"/>
                            </handlerContainer>
                            <mqListener queue="ABC.getme2" suggExpiry="30" minExpiry="4" maxExpiry="500" copyMessageId="true"/>
                    </service>

XML Structure:

     <?xml version="1.0" encoding="UTF-8"?>
        <deploymentconfig xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
                <configfile>sample.xml</configfile>
                <exceptionsFilterConfigFile>asdasd.xml</exceptionsFilterConfigFile>
                <keyInfoConfigFile>asdasd.xml</keyInfoConfigFile>
                <services>

    <service name="GETME" min="1" max="10" idleTime="300" backend="ABC">
                            <handlerContainer className="com.abc.xyz.wqere.abcqwere">
                            <handler className="com.abc.xyz.qweqweqwe.werwerwerwer"/>
                            </handlerContainer>
                            <mqListener queue="ABC.getme" suggExpiry="30" minExpiry="4" maxExpiry="500" copyMessageId="true"/>
                    </service>

    <service name="GETME2" min="1" max="10" idleTime="300" backend="ABC">
                            <handlerContainer className="com.abc.xyz.wqere.abcqwere">
                            <handler className="com.abc.xyz.qweqweqwe.werwerwerwer"/>
                            </handlerContainer>
                            <mqListener queue="ABC.getme2" suggExpiry="30" minExpiry="4" maxExpiry="500" copyMessageId="true"/>
                    </service>
        . . . .a lot of services like this . . . .
        . . . .a lot of services like this . . . .
        . . . .a lot of services like this . . . .
        . . . .a lot of services like this . . . .
        </services>
   <batchServices>
                        <batchService name="batch1">
                                <executor className="com.abc.xyz.qwer.qweqwewqe.ffdsdfsdfsdfsdf" />
                        </batchService>
                        <batchService name="batch2">
                                <executor className="com.abc.xyz.qwer.qweqwewqe.zxcsadsad" />
                        </batchService>
. . . .a lot of batch services like this . . . .
        . . . .a lot of batch services like this . . . .
        . . . .a lot of batch services like this . . . .
        . . . .a lot of batch services like this . . . .
      </batchServices>

<timerservices>
<timerservice> - a lot of timeservice
</timerservices>

  <connectionPools>
                <pool>
                        <name>asdasd</name>
                        <driver>oracle.jdbc.driver.OracleDriver</driver>
                        <url>$asdasd_URL</url>
                        <userId>$asdasd_USER</userId>
                        <password>$asdasd_PASSWORD</password>
                        <minConnections>0</minConnections>
                        <maxConnections>10</maxConnections>
                        <poolUrl>jdbc:asdsad:asdasdsad</poolUrl>
                        <testSql>select * from abc</testSql>
                </pool>

 . . a lot of pools. . .

</connectionpools>

</deploymentconfig>

I need to grep an xml block like this:

 <service name="GETME" min="1" max="10" idleTime="300" backend="ABC">
                        <handlerContainer className="com.abc.xyz.wqere.abcqwere">
                        <handler className="com.abc.xyz.qweqweqwe.werwerwerwer"/>
                        </handlerContainer>
                        <mqListener queue="ABC.getme" suggExpiry="30" minExpiry="4" maxExpiry="500" copyMessageId="true"/>
                </service>

and I only need to provide the queue name

QUEUENAME=INSERT_HERE
grep ______________ $QUEUENAME. . . 

Here is the output:

Usage : xmllint [options] XMLfiles ...
    Parse the XML files and output the result of the parsing
    --version : display the version of the XML library used
    --debug : dump a debug tree of the in-memory document
    --shell : run a navigating shell
    --debugent : debug the entities defined in the document
    --copy : used to test the internal copy implementation
    --recover : output what was parsable on broken XML documents
    --noent : substitute entity references by their value
    --noout : don't output the result tree
    --path 'paths': provide a set of paths for resources
    --load-trace : print trace of all external entites loaded
    --nonet : refuse to fetch DTDs or entities over network
    --nocompact : do not generate compact text nodes
    --htmlout : output results as HTML
    --nowrap : do not put HTML doc wrapper
    --valid : validate the document in addition to std well-formed check
    --postvalid : do a posteriori validation, i.e after parsing
    --dtdvalid URL : do a posteriori validation against a given DTD
    --dtdvalidfpi FPI : same but name the DTD with a Public Identifier
    --timing : print some timings
    --output file or -o file: save to a given file
    --repeat : repeat 100 times, for timing or profiling
    --insert : ad-hoc test for valid insertions
    --compress : turn on gzip compression of output
    --html : use the HTML parser
    --xmlout : force to use the XML serializer when using --html
    --push : use the push mode of the parser
    --memory : parse from memory
    --maxmem nbbytes : limits memory allocation to nbbytes bytes
    --nowarning : do not emit warnings from parser/validator
    --noblanks : drop (ignorable?) blanks spaces
    --nocdata : replace cdata section with text nodes
    --format : reformat/reindent the input
    --encode encoding : output in the given encoding
    --dropdtd : remove the DOCTYPE of the input docs
    --c14n : save in W3C canonical format (with comments)
    --exc-c14n : save in W3C exclusive canonical format (with comments)
    --nsclean : remove redundant namespace declarations
    --testIO : test user I/O support
    --catalogs : use SGML catalogs from $SGML_CATALOG_FILES
                 otherwise XML Catalogs starting from 
             file:///etc/xml/catalog are activated by default
    --nocatalogs: deactivate all catalogs
    --auto : generate a small doc on the fly
    --xinclude : do XInclude processing
    --noxincludenode : same but do not generate XInclude nodes
    --loaddtd : fetch external DTD
    --dtdattr : loaddtd + populate the tree with inherited attributes 
    --stream : use the streaming interface to process very large files
    --walker : create a reader and walk though the resulting doc
    --pattern pattern_value : test the pattern support
    --chkregister : verify the node registration code
    --relaxng schema : do RelaxNG validation against the schema
    --schema schema : do validation against the WXS schema
    --schematron schema : do validation against a schematron
    --sax1: use the old SAX1 interfaces for processing
    --sax: do not build a tree but work just at the SAX level

Libxml project home page: http://xmlsoft.org/
To report bugs or get some help check: http://xmlsoft.org/bugs.html

Here is the version

xmllint: using libxml version 20626

Answer 1

If you are using the latest ksh - by which I mean a recent build of ksh93 - you can actually just use it. ksh93 supports compound variable types - which are a little like a C struct - or an XML node-tree. It doesn't natively support XML at the moment - though I believe that it is planned - but it does support json right now.

I used some free online converter thing to get a json output of your sample. Still though, after cleaning up your sample a little (the p in </connectionpools> should be upper-cased, by the way) I could do:

print -j queue.services.[@name]

...and was rewarded with...

GETME

I could also do:

print -j queue.services[1].[@name]

...to get instead...

GETME2

At the linked conversion site I had to select Tab-delimited to keep it from sticking in a lot of non-breaking spaces, but other than that it seems to have come off ok. Surely there are tools you can use as easily to locally to do similar conversions.

In any case, with ksh you can read in a json tree after copying it to your clipboard just like I did, like:

read -m json queue <<<"$(xsel -bo)"

After doing that I could view the whole structure like...

print -j queue

...which printed...

{
    "batchServices": [
        {
            "@name": "batch1",
            "executor": {
                "@className": "com.abc.xyz.qwer.qweqwewqe.ffdsdfsdfsdfsdf"
            }
        },
        {
            "@name": "batch2",
            "executor": {
                "@className": "com.abc.xyz.qwer.qweqwewqe.zxcsadsad"
            }
        }
    ],
    "configfile": "sample.xml",
    "connectionPools": [
        {
            "driver": "oracle.jdbc.driver.OracleDriver",
            "maxConnections": "10",
            "minConnections": "0",
            "name": "asdasd",
            "password": "$asdasd_PASSWORD",
            "poolUrl": "jdbc:asdsad:asdasdsad",
            "testSql": "select * from abc",
            "url": "$asdasd_URL",
            "userId": "$asdasd_USER"
        }
    ],
    "exceptionsFilterConfigFile": "asdasd.xml",
    "keyInfoConfigFile": "asdasd.xml",
    "services": [
        {
            "@backend": "ABC",
            "@idleTime": "300",
            "@max": "10",
            "@min": "1",
            "@name": "GETME",
            "handlerContainer": {
                "@className": "com.abc.xyz.wqere.abcqwere",
                "handler": {
                    "@className": "com.abc.xyz.qweqweqwe.werwerwerwer"
                }
            },
            "mqListener": {
                "@copyMessageId": "true",
                "@maxExpiry": "500",
                "@minExpiry": "4",
                "@queue": "ABC.getme",
                "@suggExpiry": "30"
            }
        },
        {
            "@backend": "ABC",
            "@idleTime": "300",
            "@max": "10",
            "@min": "1",
            "@name": "GETME2",
            "handlerContainer": {
                "@className": "com.abc.xyz.wqere.abcqwere",
                "handler": {
                    "@className": "com.abc.xyz.qweqweqwe.werwerwerwer"
                }
            },
            "mqListener": {
                "@copyMessageId": "true",
                "@maxExpiry": "500",
                "@minExpiry": "4",
                "@queue": "ABC.getme2",
                "@suggExpiry": "30"
            }
        }
    ]
}

Answer 2

As mentioned in the above comment, xmllint can be used like

xmllint --xpath '//service/[@name="GETME"]' Sample.xml

The option is available at least as of libxml version 20903.

A primer on xpath syntax can be found here: http://www.w3schools.com/xpath/xpath_syntax.asp or more authoritatively https://www.w3.org/Consortium/Offices/Presentations/XSLT_XPATH/#(23)

Answer 3

With the conditions:

1. I cannot use any XML parser tool as I don't have permission , read only

2. My xmllint version does not support xpath, and I cannot update it , read only

3. I dont have xmlstarlet and cannot install it

I resorted to finding other unconventional solutions. This awk command got me what I needed

awk '
  /<service.*name=/ { f=1 ; m=0 ; res="" }
  f { res = res $0 ORS }
  f && /mqListener queue="ABC.getme2"/ { m=1 }
  /<\/service>/ { f=0 ; if (m) print res $0 }
' Sample.xml

Special thanks to @Janis for helping me here -How to implement awk range pattern in fetching an xml block when the input parameter is in the middle of the block

Answer 4

Assuming a sample xml file like this:

<services>

 <service name="GETME" min="1" max="10" idleTime="300" backend="ABC">
                        <handlerContainer className="com.abc.xyz.wqere.abcqwere">
                        <handler className="com.abc.xyz.qweqweqwe.werwerwerwer"/>
                        </handlerContainer>
                        <mqListener queue="ABC.sadasdasd" suggExpiry="30" minExpiry="4" maxExpiry="500" copyMessageId="true"/>
  </service>

  <service name="GETHIM" min="1" max="10" idleTime="300" backend="ABC">
                        <handlerContainer className="com.abc.xyz.wqere.abcqwere">
                        <handler className="com.abc.xyz.qweqweqwe.werwerwerwer"/>
                        </handlerContainer>
                        <mqListener queue="ABC.sadasdasd" suggExpiry="30" minExpiry="4" maxExpiry="500" copyMessageId="true"/>
  </service>

</services>

Solution using xmlstarlet

The solution to extract the queue value of service with name as GETME is as following:

xmlstarlet sel -t -v "/services/service[@name='GETME']/mqListener/@queue" test.xml 

Here,

-t  ==>  select template 
-v  ==>  template selected is "value", i.e. value-of <xpath>

Solution using xmllint

xmllint --xpath "/services/service[@name='GETME']/mqListener/@queue" test.xml 

Answer 5

OK, first and foremost - don't use grep. XML is not a suitable format for regex based parsing. Use an XML parser instead.

My favourite XML Parser is actually a perl module called XML::Twig.

#!/usr/bin/perl

use strict;
use warnings;

use XML::Twig;

my ($keyword, $filename) = @ARGV;

XML::Twig->new(
    'pretty_print'  => 'indented_a',
    'twig_handlers' => {
        'service[@name="'.$keyword.'"]' => sub { $_->print }
    }
)->parsefile($filename);

Invoked with myscript.pl GETME yourxml it will print any matches. (Change pretty_print to a format you like).

XML::Twig actually comes bundled with a couple of sample use cases too, such as xml_grep too, which probably does a lot of what you want anyway.

With your example XML above (slightly hacked, because it's not valid, and I've assumed that your source XML actually is, and this is a transposition error).

<service
        backend="ABC"
        idleTime="300"
        max="10"
        min="1"
        name="GETME">
      <handlerContainer className="com.abc.xyz.wqere.abcqwere">
        <handler className="com.abc.xyz.qweqweqwe.werwerwerwer" />
      </handlerContainer>
      <mqListener
          copyMessageId="true"
          maxExpiry="500"
          minExpiry="4"
          queue="ABC.getme"
          suggExpiry="30"
      />
    </service>

NB: This format is indented_a in XML::Twig. Others are available. This is at least partly an illustration of why regular expression and line based matching of XML is dangerous.