I have a line with colon separated values that I want process in awk. Lines are handled differently if variable $4 contains varible $3 at the begin.

So I wrote the expression: $4 ~ /^$3/, but unfortunaltely this does not work, it never matches. What's wrong, how can I use a variable in a regular expression pattern?

This is the full example:

green="$(tput setaf 2)"
red="$(tput setaf 1)"
yellow="$(tput setaf 3)"
normal="$(tput sgr0)"

stacks=$(docker stack ls --format='{{.Name}}')

for stack in ${stacks}; do
    status=$(docker stack ps --filter="desired-state=running" --format="{{.Name}}:{{.Node}}:{{.DesiredState}}:{{.CurrentState}}:{{.Error}}" ${stack})
    if test -z "$status"; then
        echo "${red}$stack$: disabled${normal}"
        awk -F: '                                                                            
            $4 ~ /^$3/ {print "GOOD '"${green}"'" $1 ": " $4 "'"${normal}"'"}                
            $4 !~ /^$3/ {print "BAD '"${yellow}"'" $1 ": " $3 " ≠ " $4 $5 "'"${normal}"'"}   
        ' <<<${status}

Result is always BAD, e.g. here, line:

bind_bind.1:urknall:Running:Running 18 hours ago:

should print GOOD, but prints:

BAD bind_bind.1: Running ≠ Running 18 hours ago

/^$3/ is a regular expression that is guaranteed to never match as it matches on records that have 3 after the end of the record (the $ regular expression anchor operator matches at the end of the subject, not to be confused with the $ awk operator that is used to dereference fields by number).

To test whether the third field occurs in the beginning of the fourth field, one could do either a regular expression match with match(), which will return the start position of the match (or -1 if no match was found):

awk -F ':' 'match($4, $3) == 1 { ..."GOOD"... ; next } { ..."BAD"... }'

or, for a string comparison,

awk -F ':' 'substr($4, 1, length($3)) == $3 { ..."GOOD"... ; next } { ..."BAD"... }'
  • The substr solution does exactly, what I tried to achieve, so I marked this as accepted answer, even though other answer also provide good approaches. – Marc Wäckerlin Apr 15 '18 at 10:35

You can put the regex on the right side of a ~ in a string, it doesn't have to be a /.../ construct. (The difference might have been related to parsing the RE at runtime or at compile time, but I'm not sure.) And remember that in awk, the dollar sign doesn't mean variable expansion like in shell or Perl, so you need to concatenate $3 to the rest of the string:

The first one matches, the second doesn't:

$ echo 'foo fo+' |awk '$1 ~ "^" $2'
foo fo+
$ echo 'foo o+' |awk '$1 ~ "^" $2'

/^$2/ is taken as a regex with a literal $2 in it, and the $ is the end-of-line anchor. Since you can't have anything after the EOL, the RE can never match.

  • I was wondering how /^$2/ would actually be interpreted. Perl would probably interpolate $2 but awk is awfully opaque about what it does with it. – Kusalananda Feb 2 '18 at 11:30
  • 1
    @Kusalananda, for /^$2/, there aren't really ambiguities. There's no way the $ operator ($2 is the $ operator applied to 2, it's not a variable) would be invoked inside /.../, like there's no way /1+2/ would invoke the + addition operator. There can be ambiguities with the / operator. For instance print 1 /2/ 3 is not 1 concatenated with the result of the matching of the 2 regx against $0 concatenated with 3 (for which you'd need 1 (/2/) 3 but 1 divided by 2 divided by 3. And there are ambiguities with backslash. – Stéphane Chazelas Feb 2 '18 at 11:41
  • @StéphaneChazelas Much appreciated. I believe that I might have expected /$2/ to cause some sort of error or warning since it's impossible for it to match, but that might have been expecting too much magic from the underlying regexp library... – Kusalananda Feb 2 '18 at 11:46
  • 3
    @Kusalananda, the $2 ERE is a valid POSIX ERE that is guaranteed to never match. – Stéphane Chazelas Feb 2 '18 at 11:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.