Objective: 1) Return a name value from an <a> and 2) assess a cell value from a number of HTML table rows, conditionally returning 0 or 1

I do curl -s http://[server-fqdn]/stats/servers?t=db | grep 'tr class="server" which returns a varying number of rows, as illustrated in the snippet. In this case, I list db servers.

Row snippet (NB: full row character count is between 2,000 and 2,150 characters)

<tr class="server"><td class=val><a name="srv_backend_3306/server"></a></td> [cut away] <td class=val>1d3h UP</td>

Challenge: Extract the value of name in the first <a> tag and assess the content of the <td class=val></td> (if it contains the word UP, return 0, else return 1).

I can extend my above described curl command, adding | cut -d\> -f3 | cut -d\" -f2 and I get the name - but how to get that other result? I have no unique identifier for that cell and using cut doesn't quite cut it due to dynamics in the table generation.

  • 3
    If the HTML is valid XML then you can use something like xmlstarlet to pick out the element value directly
    – roaima
    Dec 17 '20 at 9:51
  • There's also pup which is aimed at HTML
    – muru
    Dec 17 '20 at 10:03
  • You can check my answer below :)
    – Reda Salih
    Dec 17 '20 at 21:48

Here is a little snippet that was inspired by a StackOverflow answer that you can use with native bash, you will have one function to read the content of your input and then another to parse the dom content :

read_dom () {
    local IFS=\>
    read -d \< ENTITY CONTENT
    local ret=$?
    TAG_NAME=${ENTITY%% *}
    return $ret

parse_dom () {
    if [[ $TAG_NAME == "a" ]] ; then
    eval local $ATTRIBUTES
        ac=`cut -d "=" -f2  <<< "$ATTRIBUTES" | tr -d '"'`
    if [[ $TAG_NAME == "td" && "$(cut -d= -f1 <<< $ATTRIBUTES)" == "class" && $CONTENT == *"UP"*  ]] ; then

while read_dom; do
done  <<< "$(curl -s http://[server-fqdn]/stats/servers?t=db | grep 'tr class="server")"
echo "<a> tag content : $ac"
echo "return value for <td> check : $cr"

Output :

<a> tag content : srv_backend_3306/server
return value for <td> check : 0
  • I like this approach, but I managed to come up with a solution, that works for me.
    – DavDav
    Dec 18 '20 at 9:45

Found a solution - it does the job, but feel free to suggest improvements ;)

curl -s http://[server-fqdn]/stats/servers?t=db | grep 'tr class="server"' > hastats.html
for i in `cat hastats.html | grep 'tr class="server"' |  cut -d\> -f3 | cut -d\" -f2` ; do
grep $i hastats.html | ( ! grep -P " UP" -q)
echo $i $?

This returns e.g. srv_backend_3306 1 for that server, which means that it was established that in this particular row, there was an occurence of the word UP with a whitespace in front. In this context, I am certain that this particular word will either be absent, or be present exactly one time for each of these particular rows.

The ! grep -P ' UP' -q negates the returned error code, which normally would be '0' for 'occurence was found' - we want '1' returned for a particular reason.

Thanks for constructive input.

  • A few comments: i) avoid for i in $(cat file) (see Bash pitfall #1). You don't need it at here anyway: grep 'tr class="server"' hastats.html | cut .... ii) this will break very easily. What if you have .foo tr { ...} in embedded css? What if you have <div name=footr class="server"? iii) You only have lines matching the search pattern in hastats.html, why grep it again? iv) there is no need for -P in your grep. It just makes the code less portable and doesn't add anything.
    – terdon
    Dec 18 '20 at 13:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.