2

Suppose I have used awk to get the value of two different columns:

Say the output is:

abc

def

Now, I want to use both abc and def as patterns for grep, using a pipeline with the previous command. Is there any way to do this?

For more clarification:

I have a pdf file and I am searching for a keyword the file using :

pdftotext 'filename.pdf' - | grep 'pattern'

Now if any match is found, I want to use the first and second column of the result as parameters for grep in pipeline with the previous command. I can get the first two columns using 

pdftotext 'filename.pdf' - | grep 'pattern' | awk '{ print $1, "\t"$2 }'

Now how will I pass these two values again (2nd time) as patterns after grep command in pipeline with the above one?

Improve this question
2
  • Not sure what you mean. Please provide an example. – glenn jackman Sep 13 '15 at 14:22
  • I have edited the post. Hope you will understand now @glenn jackman – Black Sep 13 '15 at 14:32
5

You could do that with process substitution as follows:

grep -f <(awk '{print $1 "\n" $3}' filename) otherfile

This will have grep read its patterns from a file (the -f flag), which in this case is actually the output of the process awk...filename which prints the patterns one per line. Then grep searchs for those patterns in otherfile

Though perhaps less efficient if you really want to do it in a pipeline you could do it with grep reading its patterns from stdin like

awk '{print $1 "\n" $3}' filename | grep -f - otherfile

Edit: seeing your question's edit about using grep and then awk, you can let awk do the pattern match for you by doing:

grep -Ff <(awk '/pattern/ {print $1 "\n" $3}' <(pdftotext 'filename.pdf' -)) otherfile

or as a pipeline:

pdftotext 'filename.pdf' - | awk '/pattern/ {print $1 "\n" $3}' | grep -Ff - otherfile
Improve this answer
11
  • if filename=abc.pdf and otherfile=def.pdf, I have used the following command grep -f <(awk '{print $1 "\n" $3}' abc.pdf) def.pdf but getting error saying: grep: Unmatched [ or [^ @Eric Renouf – Black Sep 13 '15 at 15:05
  • @Black that probably means the strings being returned from awk are things that look like regular expressions to grep. You can add the -F flag to grep to tell it not to interpret the strings as regex, instead they are Fixed strings – Eric Renouf Sep 13 '15 at 16:29
  • what will be the final shell command if I want to use this in pipeline with the command I stated at last in my post? @Eric Renouf – Black Sep 13 '15 at 16:31
  • @Black see my latest update – Eric Renouf Sep 13 '15 at 16:34
  • according to you will I apply awk on input first? But I wanted to use the two return values from pdftotext 'filename.pdf' - | grep 'pattern' | awk '{ print $1, "\t"$2 }' to use again (2nd time) as patterns after grep in pipeline with this same command @Eric Renouf – Black Sep 13 '15 at 16:45
4

Many shells support command substitution so that for example

p=$(awk '{ print $1,$3 }' filename)

captures the output of the command awk '{ print $1,$3 }' filename into variable p. Note that your cat is redundant here: awk can read directly from filename.

However, I suspect what you really want to do is process successive lines of awk's output and parse them into separate variables. If that's the case, you can likely do something such as

awk '{print $1,$3}' filename | while read -r p1 p2; do grep -F "$p1" otherfile | grep -F "$p2"; done

to find lines in otherfile that match both fields $1 and $3 from filename.

Improve this answer
3
  • getting error saying: grep: Unmatched [ or [^ @steeldriver – Black Sep 13 '15 at 15:07
  • @Black please edit your question to include a Minimal, Complete, and Verifiable example: your current example using abc and def doesn't contain any special characters such as [. You may need to use grep -F to enforce fixed string matching rather than regular expression matching. – steeldriver Sep 13 '15 at 15:32
  • I have edited the post. Hope you will now understand my requirements @steeldriver – Black Sep 13 '15 at 15:51
0

This can be done using command substitution.

Quoting from here:

Command substitution allows the output of a command to be substituted in place of the command name itself. Command substitution shall occur when the command is enclosed as follows:

$(command)

or (backquoted version):

`command`

Improve this answer
0

You can learn how to use xargs to pass and define parameters.

cat filename|awk '{print $1,"\n"$3}'|xargs -I {} grep {} somefile

xargs -I {} replace the parameter with "{}" to let grep use it

Improve this answer
2
  • If my filename is abc and I am using cat abc | awk '{print $1,"\n"$3}'| xargs -I {} grep {} def ,but is not showing any result @Shellmode – Black Sep 13 '15 at 16:28
  • xarg -I {} makes abc & def both to be "{}", and you can use the "{}" to be the pattern of grep. Exmaple: If you want to find lines in file named test.txt match the pattern abc or def. And the pattern is the output of cat pattern. You can have a try: cat pattern | xargs -I {} sh -c 'grep $1 test.txt' --{} – Shellmode Sep 14 '15 at 1:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.