76

Having the following in one of my shell functions:

function _process () {
  awk -v l="$line" '
  BEGIN {p=0}
  /'"$1"'/ {p=1}
  END{ if(p) print l >> "outfile.txt" }
  '
}

, so when called as _process $arg, $arg gets passed as $1, and used as a search pattern. It works this way, because shell expands $1 in place of awk pattern! Also l can be used inside awk program, being declared with -v l="$line". All fine.

Is it possible in same manner give pattern to search as a variable?

Following will not work,

awk -v l="$line" -v search="$pattern" '
  BEGIN {p=0}
  /search/ {p=1}
  END{ if(p) print l >> "outfile.txt" }
  '

,as awk will not interpret /search/ as a variable, but instead literally.

3
  • 1
    What you're searching for is not text that matches a "pattern", it's text that matches either a string or a regular expression. See how-do-i-find-the-text-that-matches-a-pattern for why that matters and why you shouldn't use the word "pattern" in this context.
    – Ed Morton
    May 26, 2021 at 15:23
  • 1
    See also how-do-i-use-shell-variables-in-an-awk-script for a comprehensive answer to the question of how to pass the value of shell variables or other values to an awk script.
    – Ed Morton
    May 26, 2021 at 16:05
  • I've been reading a lot of info about awk and passing variables, etc. topics so apologies if I am a bit confused, but, although above answers are very good, IMO, fail to address one of OPs questions, whether it is possible or not to use "/search/" as variable. To me it sounds it is not possible, but I fail to see why or where is this stated Aug 10 at 9:55

5 Answers 5

56

Use awk's ~ operator, and you don't need to provide a literal regex on the right-hand side:

function _process () {
    awk -v l="$line" -v pattern="$1" '
        $0 ~ pattern {p=1; exit} 
        END {if(p) print l >> "outfile.txt"}
    '  
}

Here calling exit upon the first match as we don't need to read the rest. You don't even need awk, grep would be enough and likely more efficient and avoid the problem of awk's -v var='value' doing backslash processing:

function _process () {
    grep -qe "$1" && printf '%s\n' "$line"
}

Depending on the pattern, you may want grep -Eqe "$1"

8
  • This is exactly what solves this in a way I wanted (1st example), because it keeps the semantics, which was my goal. Thanks.
    – branquito
    Mar 21, 2014 at 15:30
  • 1
    I didn't note the removal of the BEGIN block: an unassigned variable is treated as 0 in a numeric context or the empty string otherwise. So, an unassigned variable will be false in if (p) ... Mar 21, 2014 at 15:35
  • yes I noticed, it needs to be set on BEGIN block to zero each time, as it serves as a switch. But interestingly I tried now script using $0 ~ pattern, and it does not work, however with /'"$1"'/ it does work!? :O
    – branquito
    Mar 21, 2014 at 15:42
  • maybe it has something to do with the way $line is retrieved, pattern search is done on the output of whois $line, $line coming from file in a WHILE DO block.
    – branquito
    Mar 21, 2014 at 15:53
  • Please show the contents of $line -- do it in your question for proper formatting. Mar 21, 2014 at 15:57
22
awk  -v pattern="$1" '$0 ~ pattern'

Has an issue in that awk expands the ANSI C escape sequences (like \n for newline, \f for form feed, \\ for backslash and so on) in $1. So it becomes an issue if $1 contains backslash characters which is common in regular expressions (with GNU awk 4.2 or above, values that start with @/ and end in /, are also a problem). Another approach that doesn't suffer from that issue is to write it:

PATTERN=$1 awk '$0 ~ ENVIRON["PATTERN"]'

How bad it's going to be will depend on the awk implementation.

$ nawk -v 'a=\.' 'BEGIN {print a}'
.
$ mawk -v 'a=\.' 'BEGIN {print a}'
\.
$ busybox awk -v 'a=\.' 'BEGIN {print a}'
.
$ gawk -v 'a=\.' 'BEGIN {print a}'
gawk: warning: escape sequence `\.' treated as plain `.'
.
$ gawk5.0.1 -v 'a=@/foo/' BEGIN {print a}'
foo

All awks work the same for valid escape sequences though:

$ a='\\-\b' awk 'BEGIN {print ENVIRON["a"]}' | od -vtx1 -tc
0000000  5c  5c  2d  5c  62  0a
          \   \   -   \   b  \n
0000006

(content of $a passed as-is)

$ awk -v a='\\-\b' 'BEGIN {print a}' | od -vtx1 -tc
0000000  5c  2d  08  0a
          \   -  \b  \n
0000004

(\\ changed to \ and \b changed to a backspace character).

10
  • So you are saying that if pattern was for example \d{3} to find three digits, that wouldn't work as expected, if I understood you well?
    – branquito
    Mar 21, 2014 at 16:24
  • 2
    for \d which is not a valid C escape sequence, that depends on your awk implementation (run awk -v 'a=\d{3}' 'BEGIN{print a}' to check). But for \` or \b, yes definitely. (BTW, I don't know of any awk implementations that understands \d` as meaning a digit). Mar 21, 2014 at 16:30
  • it says: awk warning - escape sequence \d' treated as plain d' d{3}, so I guess I would have a problem in this case?
    – branquito
    Mar 21, 2014 at 16:34
  • 1
    Sorry, my bad, I had a typo in my answer. The name of then environment variable has to match ENVIRON["PATTERN"] for the PATTERN environment variable. If you want to use a shell variable, you need to export it first (export variable) or use the ENV=VALUE awk '...ENVIRON["ENV"]' env-var passing syntax as in my answer. Mar 21, 2014 at 17:03
  • 2
    Because you need to export a shell variable for it to be passed in the environment to a command. Mar 21, 2014 at 17:20
5

Try something like:

awk -v l="$line" -v search="$pattern" 'BEGIN {p=0}; { if ( match( $0, search )) {p=1}}; END{ if(p) print l >> "outfile.txt" }'
3
  • If this behaves same as /regex/ in terms of finding pattern, this could be a nice solution. I will try.
    – branquito
    Mar 21, 2014 at 15:22
  • 1
    The quick tests I ran seemed to work the same, but I won't even begin to guarantee it... :) Mar 21, 2014 at 15:24
  • But what if $pattern (and therefore search) contains something that can be treated as a part of regex, like [? And I want to search for a literal match?
    – A S
    Nov 28, 2021 at 4:40
-1

No, but you can simply interpolate the pattern into the double-quoted string you pass to awk:

awk -v l="$line" "BEGIN {p=0}; /$pattern/ {p=1}; END{ if(p) print l >> \"outfile.txt\" }"

Note that you now have to escape the double-quoted awk literal, but it is still the simplest way of accomplishing this.

4
  • Is this way safe if $pattern contains spaces, my example from above will work as $1 is protected with "$1" double quotes, however not shure what happens in your case.
    – branquito
    Mar 21, 2014 at 15:14
  • 2
    Your original example ends the single-quoted string at the second ', then protects the $1 via double quotes and then tacks another single-quoted string for the second half of the awk program. If I understand correctly, this should have exactly the same effect as protecting the $1 via the outer single quotes - awk never sees the double quotes that you put around it. Mar 21, 2014 at 15:26
  • 5
    But if $pattern contains ^/ {system("rm -rf /")};, then you're in big trouble. Mar 21, 2014 at 16:17
  • is that downside of this approach only, having all wrapped in "" ?
    – branquito
    Mar 21, 2014 at 16:27
-3

You could use the eval function which resolves in this example the nets variable before the awk is run.

nets="searchtext"
eval "awk '/"${nets}"/'" file.txt

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.