268

Say I have a file:

# file: 'test.txt'
foobar bash 1
bash
foobar happy
foobar

I only want to know what words appear after "foobar", so I can use this regex:

"foobar \(\w\+\)"

The parenthesis indicate that I have a special interest in the word right after foobar. But when I do a grep "foobar \(\w\+\)" test.txt, I get the entire lines that match the entire regex, rather than just "the word after foobar":

foobar bash 1
foobar happy

I would much prefer that the output of that command looked like this:

bash
happy

Is there a way to tell grep to only output the items that match the grouping (or a specific grouping) in a regular expression?

  • 4
    for those who do not need grep: perl -lne 'print $1 if /foobar (\w+)/' < test.txt – vault Dec 19 '16 at 11:56
308

GNU grep has the -P option for perl-style regexes, and the -o option to print only what matches the pattern. These can be combined using look-around assertions (described under Extended Patterns in the perlre manpage) to remove part of the grep pattern from what is determined to have matched for the purposes of -o.

$ grep -oP 'foobar \K\w+' test.txt
bash
happy
$

The \K is the short-form (and more efficient form) of (?<=pattern) which you use as a zero-width look-behind assertion before the text you want to output. (?=pattern) can be used as a zero-width look-ahead assertion after the text you want to output.

For instance, if you wanted to match the word between foo and bar, you could use:

$ grep -oP 'foo \K\w+(?= bar)' test.txt

or (for symmetry)

$ grep -oP '(?<=foo )\w+(?= bar)' test.txt
  • 3
    How you do it if your regex has more than a grouping? (as the title implied?) – barracel Mar 21 '13 at 7:52
  • 4
    @barracel: I don't believe you can. Time for sed(1) – camh Mar 22 '13 at 22:51
  • 1
    @camh I have just tested that grep -oP 'foobar \K\w+' test.txt outputs nothing with the OP's test.txt. The grep version is 2.5.1. What could be wrong ? O_O – SOUser Jul 24 '14 at 14:19
  • @XichenLi: I can't say. I just built v2.5.1 of grep (it's pretty old - from 2006) and it worked for me. – camh Jul 25 '14 at 10:18
  • @SOUser: I experienced the same - outputs nothing to file. I submitted the edit request to include '>' before the filename to send output as this worked for me. – rjchicago Dec 15 '16 at 21:40
36

Standard grep can't do this, but recent versions of GNU grep can. You can turn to sed, awk or perl. Here are a few examples that do what you want on your sample input; they behave slightly differently in corner cases.

Replace foobar word other stuff by word, print only if a replacement is done.

sed -n -e 's/^foobar \([[:alnum:]]\+\).*/\1/p'

If the first word is foobar, print the second word.

awk '$1 == "foobar" {print $2}'

Strip foobar if it's the first word, and skip the line otherwise; then strip everything after the first whitespace and print.

perl -lne 's/^foobar\s+// or next; s/\s.*//; print'
  • Awesome! I thought I may be able to do this with sed, but I haven't used it before and was hoping I could use my familiar grep. But the syntax for these commands actually looks very familiar now that I am familiar with vim-style search & replace + regexes. Thanks a ton. – Cory Klein May 19 '11 at 23:51
  • 1
    Not true, Gilles. See my answer for a GNU grep solution. – camh May 20 '11 at 1:33
  • 1
    @camh: Ah, I didn't know GNU grep now had full PCRE support. I've corrected my answer, thanks. – Gilles May 20 '11 at 7:14
  • 1
    This answer is especially useful for embedded Linux since Busybox grep doesn't have PCRE support. – Craig McQueen Mar 17 '16 at 0:12
  • Obviously there are multiple ways to accomplish the same task presented, however, if the OP ask for grep usage, why you answer something else? Also, your first paragraph is incorrect: yes grep can do it. – fcm Mar 11 at 13:31
27
    sed -n "s/^.*foobar\s*\(\S*\).*$/\1/p"

-n     suppress printing
s      substitute
^.*    anything before foobar
foobar initial search match
\s*    any white space character (space)
\(     start capture group
\S*    capture any non-white space character (word)
\)     end capture group
.*$    anything after the capture group
\1     substitute everything with the 1st capture group
p      print it
  • 1
    +1 for the sed example, seems like a better tool for the job than grep. One comment, the ^ and $ are extraneous since .* is a greedy match. However, including them might help clarify the intent of the regex. – Tony May 30 '18 at 21:22
15

Well, if you know that foobar is always the first word or the line, then you can use cut. Like so:

grep "foobar" test.file | cut -d" " -f2
  • The -o switch on grep is widely implemented (moreso than the Gnu grep extensions), so doing grep -o "foobar" test.file | cut -d" " -f2 will increase the effectiveness of this solution, which is more portable than using lookbehind assertions. – dubiousjim Apr 19 '12 at 21:04
  • I believe that you would need grep -o "foobar .*" or grep -o "foobar \w+". – G-Man Apr 14 '18 at 7:20
8

If PCRE is not supported you can achieve the same result with two invocations of grep. For example to grab the word after foobar do this:

<test.txt grep -o 'foobar  *[^ ]*' | grep -o '[^ ]*$'

This can be expanded to an arbitrary word after foobar like this (with EREs for readability):

i=1
<test.txt egrep -o 'foobar +([^ ]+ +){'$i'}[^ ]+' | grep -o '[^ ]*$'

Output:

1

Note the index i is zero-based.

5

pcregrep has a smarter -o option that lets you choose which capturing groups you want output.  So, using your example file,

$ pcregrep -o1 "foobar (\w+)" test.txt
bash
happy
4

Using grep is not cross-platform compatible, since -P/--perl-regexp is only available on GNU grep, not BSD grep.

Here is the solution using ripgrep:

$ rg -o "foobar (\w+)" -r '$1' <test.txt
bash
happy

As per man rg:

-r/--replace REPLACEMENT_TEXT Replace every match with the text given.

Capture group indices (e.g., $5) and names (e.g., $foo) are supported in the replacement string.

Related: GH-462.

2

I found the answer of @jgshawkey very helpful. grep is not such a good tool for this, but sed is, although here we have an example that uses grep to grab a relevant line.

Regex syntax of sed is idiosyncratic if you are not used to it.

Here is another example: this one parses output of xinput to get an ID integer

⎜   ↳ SynPS/2 Synaptics TouchPad                id=19   [slave  pointer  (2)]

and I want 19

export TouchPadID=$(xinput | grep 'TouchPad' | sed  -n "s/^.*id=\([[:digit:]]\+\).*$/\1/p")

Note the class syntax:

[[:digit:]]

and the need to escape the following +

I assume only one line matches.

  • This is exactly what I was trying to do. Thanks! – James May 12 at 0:07
  • Slightly simpler version without the extra grep, assuming 'TouchPad' is to the left of 'id' : echo "SynPS/2 Synaptics TouchPad id=19 [slave pointer (2)]" | sed -nE "s/.*TouchPad.+id=([0-9]+).*/\1/p" – Amit Naidu May 19 at 5:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.