Group matching with grep includes extra characters

Question

I wanted to extract some text with regex in bash, so I decided to try the following simple example out.

echo "abc def ghi" | grep -Po " \K(.*?) "

I was expecting to get a "def", but to my surprise a "def " (with a final extra space) was what I got.

I'm interested in understanding why grep also includes the extra space at the end and how to get rid of it. I know I could post-process the result with another line but I'm interested in solving this with grep.

Answer 1

In short:

\K

causes grep to keep everything prior to the \K and not include it in the match. It does not affect what comes after the \K().

This might be enough:

" \K(.+)(?= )"

Where (?= ) is a non capturing group.

or perhaps better:

" \K([^ ]+)(?= )"
" \K(\w+)(?= )"

or similar.

Answer 2

A BRE doing what you're trying to in sed might look like:

sed 's/ *\(\([^ ]*\) *\)\{[num]\}.*/\2/'

...or as an ERE for those seds which support it such as GNU and BSD versions:

sed -E 's/ *(([^ ]*) *){[num]}.*/\2/p'

...either expression will begin its match at the first character of the [num]th group (where [num] is a positive integer) of [^ ]*not-space characters in pattern-space and continue matching until line's end.

The important thing, though, is it subgroups some matches as it does so:

• (([^ ]*) *){[num]} - this group matches as many [num] occurrences of not-space groups and any/all subsequent space characters and can be back-reference as \1.
  • {[num]} - when a pattern is matched \{[num]\} times the only reference to it that remains is the last - and so even though this group matches as many occurrences of the pattern as specified, the only reference it returns is the last.
• ([^ ]*) - the subgroup of the above group, though, matches only the subset of not-space characters matched in \1. This sub-group can be referenced in \2.
• * and .* - this matches any/all space characters leading pattern space and any/all characters following the occurrences matched in the subexpressions.
• /\2/ - this replaces all of the above with only the group referenced in \2.

Because [^ ]* and * are boolean complements and that [^ ]*U* together can describe any possible string, the above regex works universally.

For your example:

for n in 1 2 3 4
do  echo "abc def ghi" | 
    sed -E "s/ *(([^ ]*) *){$n}.*/\2/"
done | sed -n l

...prints...

abc$
def$
ghi$
$

As is, it will always print a blank-line for a specified occurrence above that asked for, but - if that is not desirable - the line can be removed from output entirely like:

sed -En 's/ *(([^ ]*) *){[num]}.*/\2/;/./p'

Taking that a little further, the substitution can be applied globally to get only every [num]th occurrence. And since * is pretty limiting, I'll do it with [[:space:]]* instead - which will match any of <space><tab><newline><vertical tab><return>.

s=
{   printf "${s:=$(printf '\r\v\t%10s')}"
    seq -s"$s" 100
} | sed -En "s/[${s:=[:space:]}]*(([^$s]*)[$s]*){21}/\2\\
/g;      /[^$s]/s/\n*$//p"

Before applying sed to it the above printf ...; seq ... bit prints a single line like:

\r\v\t          1\r\v\t          2\r\v\t          3\r\v\t...

... and so on. But applying the above sed to it gets instead:

21
42
63
84

...and no blanks follow the numbers printed.