1

I have a XML file of little huge size. I have been provided with that and all I need to do is a extract some values in between the XML tags. Since I don't have the XML parser utility available in my machines. I am looking for an alternate method.

To start with, there is a XML tag <capacity> </capacity> which repeats n number of time in the XML file and in between this XML tags there are many other different tags as well.

I have to get each occurrence of <capacity> </capacity> XML tag separately and then parse through that and extract the values under them.

<subcolumns><capacity><name>45.90</name>
<index>0</index>
<value_type>String</value_type>
<ignore_case_flag>1</ignore_case_flag>
<hidden_flag>0</hidden_flag>
<exclude_from_parse_flag>1</exclude_from_parse_flag>
</capacity>
<capacity><name>57.09</name>
<index>1</index>
<value_type>String</value_type>
<ignore_case_flag>1</ignore_case_flag>
<hidden_flag>0</hidden_flag>
<exclude_from_parse_flag>1</exclude_from_parse_flag>
</capacity>
<capacity><name>55</name>
<index>2</index>
<value_type>String</value_type>
<ignore_case_flag>1</ignore_case_flag>
<hidden_flag>0</hidden_flag>
<exclude_from_parse_flag>1</exclude_from_parse_flag>
</capacity>
</subcolumns>

So the logic which I thought was to find the first occurrence of a <capacity> </capacity> XML tag and print it to a temp file and then delete that first occurrence.

<capacity><name>45.90</name>
<index>0</index>
<value_type>String</value_type>
<ignore_case_flag>1</ignore_case_flag>
<hidden_flag>0</hidden_flag>
<exclude_from_parse_flag>1</exclude_from_parse_flag>
</capacity>

Henceforth when this is done for the second time the new pair of <capacity> </capacity> XML tag is taken into consideration. So this has to repeat for multiple times until the last <capacity> </capacity> tag is found. And each time this part is extracted the data will be changing and that can be extracted.

Now all I want is to select the first occurrence of <capacity> </capacity> XML tag from the master XML file & print it to temp file and delete that part.

And this is what I tried and nothing worked for me.

sed -n '2,${/<capacity>\(.*\)<\/capacity>/\1/p;q;}' "<input XML file>" >> temp.txt

My further idea is to take that temp file for processing and extract the values which I need to under the capacity tags. For which I have already written the logic and it is working fine.

  • Will <capacity> and </capacity> always be on separate lines or can they appear on the same line? Can you have nested tags? I mean one <capacity> </capacity> inside another <capacity> </capacity>? Is there any specific reason why you are doing it this way instead of using an XML parser? – terdon Jun 5 '17 at 15:48
  • This is easy to do in a single pass with an xml parser. However, it isn't possible to answer your question in that context, since you didn't disclose what data you want to extract and in what format. – Satō Katsura Jun 5 '17 at 15:57
  • @terdon a. <capacity> and </capacity>are always separate. b.Nope even nested tags are present in my XML file. c. The machine which I am using nor where I will be deploying this script once done does not have XML Parser. And I don't have the privilege to install this package as well. – ramp Jun 5 '17 at 17:50
  • @SatoKatsura a.As commented above for some reason I don't have the XML parser available in my machine. b. I need to extract the values of the tag name,index and hidden_value for now . And the reason for not disclosing that is I have the code already done for this. – ramp Jun 5 '17 at 18:12
5

Using XML parsers is the right way for manipulating XML documents.

xmlstarlet solution:

xmlstarlet sel -t -c '//capacity[1]' -n yourxml > temp.txt 
&& xmlstarlet ed -d '//capacity[1]' yourxml > tmp.xml && mv tmp.xml yourxml 

cat temp.txt
<capacity><name>45.90</name>
<index>0</index>
<value_type>String</value_type>
<ignore_case_flag>1</ignore_case_flag>
<hidden_flag>0</hidden_flag>
<exclude_from_parse_flag>1</exclude_from_parse_flag>
</capacity>

  • xmlstarlet sel -t -c '//capacity[1]' -n yourxml > temp.txt - extracts the first capacity tag declaration and redirects the output to temp.txt

  • xmlstarlet ed -d '//capacity[1]' yourxml > tmp.xml - deletes the first capacity tag from the document (via -d delete action) and redirects the modified document content to temporary file tmp.xml

  • mv tmp.xml yourxml - replace the initial xml document with its modified version

  • Thanks for the your idea!. I was able to somehow get a server with admin privileges and installed the xmlstarlet utility. Now to make this more stronger I made a if loop such that all the <capacity> blocks are captured one by one. Now the problem when I use the $i variable along the xmlstarlet command it is printing the entire file rather than for occurrence. Please find the below snippet. #!/bin/bash set -x capacity_count=grep -c "<capacity>" sample1.xml` for ((i=1;i<=$capacity_count;i++)); do xmlstarlet sel -t -c '//capacity["$i"]' -n sample1.xml done – ramp Jun 7 '17 at 13:14
  • No able to get the formatting in order to paste the code. – ramp Jun 7 '17 at 13:18
2

Since you don't have an xml parser , give a try to this one with gnu awk:

$ awk -v RS="<subcolumns>|</capacity>" 'NR==2{gsub(/^\n/,"");print $0 RT;exit}' file.xml
<capacity><name>45.90</name>
<index>0</index>
<value_type>String</value_type>
<ignore_case_flag>1</ignore_case_flag>
<hidden_flag>0</hidden_flag>
<exclude_from_parse_flag>1</exclude_from_parse_flag>
</capacity>

Why this works:
in awk you can set a custom record / line separator. In our example we use either <subcolumns> or </capacity>

As a result with a customized record separator we can achieve this format:

$ awk -v RS="<subcolumns>|</capacity>" '{gsub(/^\n/,"");print NR,$0 RT}' file.xml
1 <subcolumns>
2 <capacity><name>45.90</name>
<index>0</index>
<value_type>String</value_type>
<ignore_case_flag>1</ignore_case_flag>
<hidden_flag>0</hidden_flag>
<exclude_from_parse_flag>1</exclude_from_parse_flag>
</capacity>
3 <capacity><name>57.09</name>
<index>1</index>
<value_type>String</value_type>
<ignore_case_flag>1</ignore_case_flag>
<hidden_flag>0</hidden_flag>
<exclude_from_parse_flag>1</exclude_from_parse_flag>
</capacity>
4 <capacity><name>55</name>
<index>2</index>
<value_type>String</value_type>
<ignore_case_flag>1</ignore_case_flag>
<hidden_flag>0</hidden_flag>
<exclude_from_parse_flag>1</exclude_from_parse_flag>
</capacity>
5 </subcolumns>

PS: NR = Number or Record/Line. RT=Record Separator used.

Accordingly you can print any <capacity></capacity> block by adjusting the NR number:

$ awk -v RS="<subcolumns>|</capacity>" 'NR==4{gsub(/^\n/,"");print $0 RT}'
<capacity><name>55</name>
<index>2</index>
<value_type>String</value_type>
<ignore_case_flag>1</ignore_case_flag>
<hidden_flag>0</hidden_flag>
<exclude_from_parse_flag>1</exclude_from_parse_flag>
</capacity>

Accordingly, if you need to skip a <capacity> section (i.e first capacity section) you can just do :

$ awk -v RS="<subcolumns>|</capacity>" '{gsub(/^\n/,"")}NR!=2{print $0 RT}'
<subcolumns>
<capacity><name>57.09</name>
<index>1</index>
<value_type>String</value_type>
<ignore_case_flag>1</ignore_case_flag>
<hidden_flag>0</hidden_flag>
<exclude_from_parse_flag>1</exclude_from_parse_flag>
</capacity>
<capacity><name>55</name>
<index>2</index>
<value_type>String</value_type>
<ignore_case_flag>1</ignore_case_flag>
<hidden_flag>0</hidden_flag>
<exclude_from_parse_flag>1</exclude_from_parse_flag>
</capacity>
</subcolumns>
0
chunk=2; # specify the tag chunk number here which must be > 0
perl -l -0777ne "print((m{(<capacity>.*?</capacity>)}sg)[${chunk:-1}-1])" yourxmlfile

Results

<capacity><name>57.09</name>
<index>1</index>
<value_type>String</value_type>
<ignore_case_flag>1</ignore_case_flag>
<hidden_flag>0</hidden_flag>
<exclude_from_parse_flag>1</exclude_from_parse_flag>
</capacity>

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.