5

I wanted to extract some text with regex in bash, so I decided to try the following simple example out.

echo "abc def ghi" | grep -Po " \K(.*?) "

I was expecting to get a "def", but to my surprise a "def " (with a final extra space) was what I got.

I'm interested in understanding why grep also includes the extra space at the end and how to get rid of it. I know I could post-process the result with another line but I'm interested in solving this with grep.

2
  • 1
    Would " \K(.+)(?= )" suffice?
    – user367890
    Jan 18 '15 at 7:32
  • From my preliminary testing, yes! Jan 18 '15 at 7:33
6

In short:

\K

causes grep to keep everything prior to the \K and not include it in the match. It does not affect what comes after the \K().

This might be enough:

" \K(.+)(?= )"

Where (?= ) is a non capturing group.

or perhaps better:

" \K([^ ]+)(?= )"
" \K(\w+)(?= )"

or similar.

6
  • That is great! I keep wondering, though, if there isn't a more elegant solution at hand? In Java/C#'s Regex I wouldn't need this kind of work-arounds. Jan 18 '15 at 7:39
  • @devouredelysium: But then you are using the capturing group, or? As in $1 or \1 etc?
    – user367890
    Jan 18 '15 at 7:45
  • In C# I would refer to the first group, second, etc. I'm not aware if the same is possible using grep? Jan 18 '15 at 7:49
  • @devouredelysium: One option is to use pcregrep as in pcregrep -o1 " (\w+) ". But then again that is not grep.
    – user367890
    Jan 18 '15 at 8:02
  • Perfect! 1234567 Jan 18 '15 at 8:04
2

A BRE doing what you're trying to in sed might look like:

sed 's/ *\(\([^ ]*\) *\)\{[num]\}.*/\2/'

...or as an ERE for those seds which support it such as GNU and BSD versions:

sed -E 's/ *(([^ ]*) *){[num]}.*/\2/p'

...either expression will begin its match at the first character of the [num]th group (where [num] is a positive integer) of [^ ]*not-space characters in pattern-space and continue matching until line's end.

The important thing, though, is it subgroups some matches as it does so:

  • (([^ ]*) *){[num]} - this group matches as many [num] occurrences of not-space groups and any/all subsequent space characters and can be back-reference as \1.
    • {[num]} - when a pattern is matched \{[num]\} times the only reference to it that remains is the last - and so even though this group matches as many occurrences of the pattern as specified, the only reference it returns is the last.
  • ([^ ]*) - the subgroup of the above group, though, matches only the subset of not-space characters matched in \1. This sub-group can be referenced in \2.
  • * and .* - this matches any/all space characters leading pattern space and any/all characters following the occurrences matched in the subexpressions.
  • /\2/ - this replaces all of the above with only the group referenced in \2.

Because [^ ]* and * are boolean complements and that [^ ]*U* together can describe any possible string, the above regex works universally.

For your example:

for n in 1 2 3 4
do  echo "abc def ghi" | 
    sed -E "s/ *(([^ ]*) *){$n}.*/\2/"
done | sed -n l

...prints...

abc$
def$
ghi$
$

As is, it will always print a blank-line for a specified occurrence above that asked for, but - if that is not desirable - the line can be removed from output entirely like:

sed -En 's/ *(([^ ]*) *){[num]}.*/\2/;/./p'

Taking that a little further, the substitution can be applied globally to get only every [num]th occurrence. And since * is pretty limiting, I'll do it with [[:space:]]* instead - which will match any of <space><tab><newline><vertical tab><return>.

s=
{   printf "${s:=$(printf '\r\v\t%10s')}"
    seq -s"$s" 100
} | sed -En "s/[${s:=[:space:]}]*(([^$s]*)[$s]*){21}/\2\\
/g;      /[^$s]/s/\n*$//p"

Before applying sed to it the above printf ...; seq ... bit prints a single line like:

\r\v\t          1\r\v\t          2\r\v\t          3\r\v\t...

... and so on. But applying the above sed to it gets instead:

21
42
63
84

...and no blanks follow the numbers printed.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.