Not sure why I'm not getting this. I've been searching and testing my command for a couple hours and I'm not getting anywhere.

The text is:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?><result expand="changes,testResults,metadata,logEntries,plan,vcsRevisions,artifacts,comments,labels,jiraIssues" key="EP-ED-JOB1-174" state="Failed" lifeCycleState="Finished" number="174" ....

And I just want to pull out the ' state="Failed" ' part, it could also be ' state="Successful" '

I've tried a million variations of this:

sed '/state=".*"/p' htmlResponse.txt

But paren's, escape slashes etc seem to match the entire chunk of text. What's wrong with my regex?

  • you need to use capture groups around what you want and use substitution to print only those portion.. to avoid greedy issue, in this case you can use [^"]* instead of .*... but really, you should use xml parser instead of regex – Sundeep Oct 16 '17 at 15:39
  • If I do sed -n '/state="[^"]*/p' htmlResponse.html it still gives me back everything. – Justin Oct 16 '17 at 15:42
  • Use xmllint instead. Use the right tools for the right job. – Valentin Bajrami Oct 16 '17 at 15:48

Putting aside the obligatory "you should really be using a proper XML parser because regexes aren't powerful enough to parse XML" comment, I see two problems in your sed line:

  1. ".*" will match from the first " to the last, since . matches "
  2. The sed command /.../p prints the whole line if it matches the regex.

Here's two things I'd suggest for quick-and-dirty HTML-scraping shell scripts:

  1. Use "[^"]*" to match "quote, any number of non-quote characters, end quote"
  2. It's lots easier to use grep -o to pull out bits of a file that match a regex

So that would make your command more like:

grep -o 'state="[^"]*"'

Or, if you really must use sed:

sed -n 's/.*\(state="[^"]*"\).*/\1/p'
| improve this answer | |
  • Thanks! I went with grep as the command just looks easier to type and understand. – Justin Oct 16 '17 at 16:16

The right way is to use XML parsers like xmlstarlet:

printf 'state="%s"\n' $(xmlstarlet sel -t -v "//result/@state" -n htmlResponse.txt)

The output:

| improve this answer | |

You likely want to match the whole line and print just the matching group:

sed -r 's/.*state="([^"]*)".*/\1/' htmlResponse.txt

That actually just pulls out the Failed or Successful (without including the state= part that precedes it), which I suspect is what you want. But if you do need that, you can add it back easily, or use a slightly different regex, as in wwoods's answer.

However, as Sundeep mentions, it is not at all robust to parse HTML (or XML) with a regular expression. It's one thing to use grep or sed to search for things interactively, but if this is part of a script that needs to carry out an important task and actually work, you should parse the XML properly.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.