Not sure why I'm not getting this. I've been searching and testing my command for a couple hours and I'm not getting anywhere.

The text is:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?><result expand="changes,testResults,metadata,logEntries,plan,vcsRevisions,artifacts,comments,labels,jiraIssues" key="EP-ED-JOB1-174" state="Failed" lifeCycleState="Finished" number="174" ....

And I just want to pull out the state="Failed" part, it could also be state="Successful".

I've tried a million variations of this:

sed '/state=".*"/p' htmlResponse.txt

But paren's, escape slashes etc seem to match the entire chunk of text. What's wrong with my regex?

  • you need to use capture groups around what you want and use substitution to print only those portion.. to avoid greedy issue, in this case you can use [^"]* instead of .*... but really, you should use xml parser instead of regex
    – Sundeep
    Oct 16 '17 at 15:39
  • If I do sed -n '/state="[^"]*/p' htmlResponse.html it still gives me back everything.
    – Justin
    Oct 16 '17 at 15:42
  • Use xmllint instead. Use the right tools for the right job. Oct 16 '17 at 15:48
  • Your XML document is not well formed. Please show a complete, or at least parseable, representation of the data.
    – Kusalananda
    Feb 3 at 10:03

Putting aside the obligatory "you should really be using a proper XML parser because regexes aren't powerful enough to parse XML" comment, I see two problems in your sed line:

  1. ".*" will match from the first " to the last, since . matches "
  2. The sed command /.../p prints the whole line if it matches the regex.

Here's two things I'd suggest for quick-and-dirty HTML-scraping shell scripts:

  1. Use "[^"]*" to match "quote, any number of non-quote characters, end quote"
  2. It's lots easier to use grep -o to pull out bits of a file that match a regex

So that would make your command more like:

grep -o 'state="[^"]*"'

Or, if you really must use sed:

sed -n 's/.*\(state="[^"]*"\).*/\1/p'
  • Thanks! I went with grep as the command just looks easier to type and understand.
    – Justin
    Oct 16 '17 at 16:16

The right way is to use XML parsers like xmlstarlet:

printf 'state="%s"\n' $(xmlstarlet sel -t -v "//result/@state" -n htmlResponse.txt)

The output:


To only get the value of the attribute (from all result nodes, if there are several):

xmlstarlet sel -t -v "//result/@state" -n htmlResponse.txt

You likely want to match the whole line and print just the matching group:

sed -r 's/.*state="([^"]*)".*/\1/' htmlResponse.txt

That actually just pulls out the Failed or Successful (without including the state= part that precedes it), which I suspect is what you want. But if you do need that, you can add it back easily, or use a slightly different regex, as in wwoods's answer.

However, as Sundeep mentions, it is not at all robust to parse HTML (or XML) with a regular expression. It's one thing to use grep or sed to search for things interactively, but if this is part of a script that needs to carry out an important task and actually work, you should parse the XML properly.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.