Objective: extract numerical values from a series of span blocks embedded in a certain div block on a remote server using curl

I need to extract some numbers from a status page and have identified the div block that holds these numbers.

The format is as follows;

<div class="stats-values"><img src="some-image-name.jpg"><span>[numeric]</span> <img src="some-image-name.jpg"><span>[numeric]</span></div>

The source is all in one line. This particular div block only occurs once and is easily identifiable through the class name but may contain as many as 6 or 7 span blocks, which is what I'm after; these span blocks have no ID or class.

I have no interest in the decorative images, only the numbers inside the span blocks, which I want to output either comma -or space separated.

I had imagine that it should be something like:

curl http://webpage.example.com/status | grep "<div class=\"stats-values\">.*</div>" | grep "<span>.*</span>"

I have tried a few examples that popped up in searches on misc. fora but to no avail so far.

I'd appreciate a pointer as to structure and syntax, be it grep, sed or awk.

  • check the character encoding after download, so that "<" is not printed as &gt; – Hölderlin Nov 21 '20 at 18:56
  • 1
    Use a proper tool, an html-parser, for parsing html, not text-parsing tools, because html is not text and could escape text-processing in many unpredictable ways. – thanasisp Nov 21 '20 at 19:21
curl http://webpage.example.com/status             |\
    grep -oP '<div class="stats-values">.*?</div>' |\
    grep -oP '(?<=<span>)\[.*?\](?=</span>)'

first grep extract the related <div ...></div> block;
second grep extract the numerical parts within inner <span>[...]</span> blocks.

(?<=pattern) is PCRE positive-lookbehind extension to the GNU grep we enabled to use it with -P switch.
(?=pattern) is PCRE positive lookahead extension.

and this (?<=<span>)\[.*?\](?=</span>) means our pattern [nuerical] is exactly within those two patterns; and these are exclusive from the output and will return only the pattern inside that matched and found.

.* is greedy match (longest possible match); .*? is non-greedy (shortest possible match)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.