2

I have to parse a .xml file generated to summarize the results of a running a testSuite on some software. In a line I have, for example:

<Summary failed="10" notExecuted="0" timeout="0" pass="18065" />

This indicates the number of tests failed, not executed, and passed. I need to figure out how many tests were in the test Suite, so I'd need to add, in the case above, 10+0+18065 = 18075.

How can I do this in Bash?

3

You can use xmlstarlet for proper xml parsing.

For your issue:

total=0; \
for i in failed notExecuted pass; do \
    sum=`xmlstarlet sel -t -v "//Summary/@$i" test.xml`; \
    total=$(($sum + $total)); \
done; \
echo "Total=$total"

where test.xml is your file containing the xml data.

1
  • 1
    @farid99 - Just copy your data into a file like test.xml, copy paste the above commands on the bash prompt & you are good to go
    – neuron
    Jul 10 '15 at 17:44
3

Using perl

perl -lne 'my @a=$_=~/(\d+)/g;$sum+=$_ for @a; print $sum' file

Using awk

tr ' ' '\n' < file | 
    awk '/[0-9]+/ {gsub(/[^0-9]/, "", $0); sum+=$0} END {print sum}'

Example

% perl -lne 'my @a=$_=~/(\d+)/g;$sum+=$_ for @a; print $sum' foo
18075

% tr ' ' '\n' < foo | 
    awk '/[0-9]+/ {gsub(/[^0-9]/, "", $0); sum+=$0} END {print sum}' 
18075

% cat foo
<Summary failed="10" notExecuted="0" timeout="0" pass="18065" />
2

Simply throw away all characters that are not digits and not space:

echo '<Summary failed="10" notExecuted="0" timeout="0" pass="18065" />'|\
sed -e 's/[^0-9 ]//g'

gives

10 0 0 18065

.

The sum can be done with dc (with the timeout field filtered as requested)

echo '<Summary failed="10" notExecuted="0" timeout="0" pass="18065" />'|\
sed -e 's/timeout="[0-9]*" //' \
    -e 's/[^0-9 ]//g' \
    -e 's/^ *//' \
    -e 's/ *$//' \
    -e 's/ /+/g' \
    -e 's/^/0 /' \
    -e 's/$/pq/'|dc

.

Description

As a sed script this would look like this

s/timeout="[0-9]*" //    #remove the timeout
s/[^0-9 ]//g             #drop anything but numbers and spaces
s/^ *//                  #drop spaces at the beginning of the line
s/ *$//                  #drop spaces at the end of the line
s/ /+/g                  #replace remaining spaces with +
s/^/0 /                  #add a 0 to initialize the sum for dc
s/$/pq/                  #add print and quit command for dc

The script can be simply used with

INPUT|sed -f script.sed

. I leave it for you to apply this script with sed and dc for multiline input. The one I wrote just works on one line!

3
  • The 3rd number is "timeout" and should not be added. Would I simply use sed -d ' ' to select the first, second, and fourth partition? Or is there a better way?
    – farid99
    Jul 10 '15 at 16:22
  • 1
    you can simply add a -e 's/timeout="[0-9]*" //' as the first expression to sed.
    – ikrabbe
    Jul 10 '15 at 16:24
  • I don't think you need to drop (^ *| *$) - they shouldn't matter to dc. It's a compiler, it expects tokenized input and doesn't care about spacea of any kind. I think. That said - there have been a lot of dcs, and I've only ever got familiar w/ one. Anyway, if you wrote the sed like: sed '/[0-9]/!d;s/[^0-9]*\([0-9]*\)[^0-9]*/ \1/g you'd know most out in one go.
    – mikeserv
    Jul 10 '15 at 19:37
1

Here's another one with dc:

{   tr -cs 0-9 \\n 
    echo '[pq]sq[z2>q+l+x]s+l+x'
}   <<\IN | dc
<Summary failed="10" notExecuted="0" timeout="0" pass="18065" />
IN

dc first reads in the whole infile - after tr has squeezed every sequence of not-number chars into a single \newline - and afterward it reads in a little echoed loop macro string which tells it to add all values on its stack each to the next until fewer than two remain, at which point it just prints whatever the sum was and quits. In this case the sum is...


18075

If you have a GNU dc you might write it like:

tr -cs 0-9 \\n <in | dc -f- -e'[pq]sq[z2>q+l+x]s+l+x'

Or, if the infile is very big, you'll want to block it, too, to keep dc from buffering too much on its stack at once.

(tr -cs 0-9 \\n|xargs -n128|tr \  +)<in |
 dc -e'[pq]sq' -e'0[?z2>q+l+x]s+l+x'

...which will buffer 128 numbers at a time.

Like this:

seq -skfkridmdk 100000 |
(tr -cs 0-9 \\n|xargs -n128|tr \  +)|
 dc -e'[pq]sq' -e'0[?z2>q+l+x]s+l+x'

5000050000

If you're sure of the four per line thing, and you want to drop *timeout=* then you could do:

<in  grep '^<Summary'   |
     cut -d\" -f2,4,8   |
     tr \" \\n          |
     xargs -n512        |
     tr \  +            |
     dc -e'[?z2>q+l+x]s+'\
        -e\[pq]sq -e0l+x

Which will only tally failed,notExecuted,pass so long as their " relative positions are constant and those are the only kinds of input lines which might match ^<Summary. I tried it like:

for x in 512 4096 16384; do time \
yes $'kdkeifndjei\n<Summary failed="10" notExecuted="0" timeout="0" pass="18065" />'|
     grep '^<Summary'   |
     cut  -d\" -f2,4,8  |
     head -n1000000     |
     tr \" \\n          |
     xargs -n"$x"       |
     tr \  +            |
     dc -e'[?z2>q+l+x]s+'\
        -e\[pq]sq -e0l+x
done

...for 3 tallies of 3mil values a piece, and the results were:

18075000000
4.00s user 0.04s system 72% cpu 5.549 total
18075000000
2.82s user 0.01s system 99% cpu 2.831 total
18075000000
2.67s user 0.01s system 99% cpu 2.680 total
1

Using an XML parser such as XMLStarlet with the file given in the question:

$ xml sel -t -m '//Summary' -v '@failed+@notExecuted+@timeout+@pass' -nl file.xml
18075

If the Summary node is found is several places, there will be one line of output for each node.

On some systems, XMLStarlet is installed as xmlstarlet rather than as xml.

0

A variant of the one of @ikrabbe. Think that this should work: echo '<Summary failed="10" notExecuted="0" timeout="0" pass="18065" />'| sed -e 's/[^0-9 ]//g' | cut -d" " -f2,3,5 | tr " " "+" | bc The fields start with 2 since there is a space at the first position.

0

Building on ikrabbe's answer.

echo $(($(echo '<Summary failed="10" notExecuted="0" timeout="0" pass="18065" />' | sed -e 's/[^0-9 ]//g' -e 's/ /+/g' -e 's/[+]*$//')))

gives "18075"

Explanation of code:

  • Echo the text, then pipe it into sed for parsing using 3 sed statements.
  • 1st sed-statement - Remove all but numbers and spaces (" 10 0 0 18065 ")
  • 2nd sed-statement - Exchange spaces with addition signs ("+10+0+0+18065+")
  • 3rd sed-statement - Remove trailing plus-signs ("+10+0+0+18065"). The inbuilt bash-calculator gets angry if you give it a calculation that end with an operator)

Everything is then wrapped in this bash-function:

$(( ))

Which computes what is inside it, and thus gives you the answer in a single line of code.

This is a two line version. Just fil the string "input" with the lines you want to process.

input="<Summary failed="10" notExecuted="0" timeout="0" pass="18065" />"
echo $(($(echo $input | sed -e 's/[^0-9 ]//g' -e 's/ /+/g' -e 's/[+]*$//')))

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.