I have this syntax in a file (http response times from analog):

<thead><tr><th class="x">seconds</th><th class="R">reqs</th><th class="r">%reqs</th><th class="B">Gbytes</th><th class="b">%bytes</th></tr></thead>
<tbody><tr><td class="x">0</td><td class="R">10927</td><td class="r"> 0.47%</td><td class="B">0.01</td><td class="b"> 0.18%</td></tr>
<tr><td class="x">&lt;= 0.01</td><td class="R">1026471</td><td class="r">44.59%</td><td class="B">0.11</td><td class="b"> 1.81%</td></tr>
<tr><td class="x">0.01-0.02</td><td class="R">535390</td><td class="r">23.26%</td><td class="B">0.06</td><td class="b"> 0.95%</td></tr>
<tr><td class="x">0.02-0.05</td><td class="R">93298</td><td class="r"> 4.05%</td><td class="B">0.27</td><td class="b"> 4.29%</td></tr>


What I want to be left with is the value for seconds - so 2 characters after "x" and before the first <

And also the request length, so 2 characters after "R" and before the next subsequent <

Probably not the best exercise to get to grips with regex but that's what I'm stuck with. Any help would be incredibly useful.

Expected result:

seconds reqs 
0 10927 
&lt= 0.01 1026471 
0.01-0.02 535390 
0.02-0.05 93298
  • 1
    The implementation of your theory: sed 's/.*"x".\([^<]*\).*"R".\([^<]*\).*/\1\t\2/' file Note that I posted this for your education only. Better do not use such solution. – manatwork Mar 25 '13 at 7:05
  • 4
    I feel that this may be relevant here: stackoverflow.com/questions/1732348/… ;). No but seriously, if you need to parse HTML you might want to look at html parsers. – Kotte Mar 25 '13 at 7:08

Unless you are very certain about the format of the HTML, you have control over it, it is not critical with errors, etc. you could use regex, - but as mentioned - not recommended.

I use it frequently myself, but then usually for one time extraction of some simple data.

You could use e.g. Perl with HTML::TokeParser::Simple.

Very simplified:


use strict;
use warnings;

use HTML::TokeParser::Simple;
use HTML::Entities;
use utf8;

die "$0 [file | url]\n" unless defined $ARGV[0];

my $tp;
if ($ARGV[0] =~ /^http:\/\//) {
    $tp = HTML::TokeParser::Simple->new(url => $ARGV[0]);
} else {
    $tp = HTML::TokeParser::Simple->new(file => $ARGV[0]);

if (!$tp) {
    die "No HTML file found.\n";

# Array to store data.
my @val;
# Index
my $i = 0;

# A bit mixed code with some redundancy. 
# Could be done much simpler, - or much more safe. 
# E.g. Check for thead, tbody etc and call a sub to parse those.
# You could off course also print directly (not save to array),
# but you might want to use the data for something?
while (my $token = $tp->get_token) {
    if ($token->is_start_tag('th') && $token->get_attr('class') eq 'x') {
        $val[$i++] = $tp->get_token->as_is;
    } elsif ($token->is_start_tag('th') && $token->get_attr('class') eq 'R') {
        $val[$i++] = $tp->get_token->as_is;
    } elsif ($token->is_start_tag('td') && (
            ($token->get_attr('class') eq 'x') ||
            ($token->get_attr('class') eq 'R'))) {
        $val[$i++] = decode_entities($tp->get_token->as_is);

my @width_col = (10, 8);

if ($i > 2 && !($i % 2)) {
    $i = 0;
    printf("%*s %*s\n",
        $width_col[0], "$val[$i++]",
        $width_col[1], "$val[$i++]"
    while ($i < $#val) {
        printf("%*s %*d\n",
            $width_col[0], "$val[$i++]",
            $width_col[1], "$val[$i++]"
} else {
    die "ERR. Unable to extract data.\n"

Sample result:

$ ./extract htmlsample 
   seconds     reqs
         0    10927
   <= 0.01  1026471
 0.01-0.02   535390
 0.02-0.05    93298

As has been mentioned regex is not good for parsing html. Similar to another parse answer you can make a Ruby one-liner such as the following to do it for you. Note that it requires Nokogiri which you can install as a gem (sudo gem install nokogiri).

ruby -rnokogiri -e 'h = Nokogiri::HTML(readlines.join); h.css("tr .x").zip(h.css("tr .R")).each { |d| puts "#{d[0].content} #{d[1].content}" }' sample.html

It reads from sample.html and makes a two dimensional array that contains everything with the attribute class="x" inside a tr element paired with everything with the attribute class="R" inside a tr element. Then it prints one such pair per line. For your example the output is the following:

seconds reqs
0 10927
<= 0.01 1026471
0.01-0.02 535390
0.02-0.05 93298

This uses sed and then you can use cut to get the fields you want. This is a one-liner, but I wrote as a script file with comments for clarity.

#!/bin/sed -f
s!</*thead!<tbody!g;      # to not get caught by 'th' below
s!<t[dh][^>]*>!%%%!g;     # replace start tag 'td' or 'th' with a delimitor
s!</t[dh]>!@@@!g;         # replace end tag 'td' or 'th' with a delimitor
s/<[^>]*>//g;             # delete any other tags
s/%%%\([^@]*\)@@@/\1 /g;  # get text between start and stop delimitors with a space
s/ $//                    # remove trailing space

Calling this as:

$ sed -f glean.sed test.html
seconds reqs %reqs Gbytes %bytes
0 10927  0.47% 0.01  0.18%
&lt;= 0.01 1026471 44.59% 0.11  1.81%
0.01-0.02 535390 23.26% 0.06  0.95%
0.02-0.05 93298  4.05% 0.27  4.29%

Then you can use what you like to get the first two fields (as I suggest with cut).

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.