5

I have a pattern variable with below value:

\"something//\\anotherthing'

and a file with below contents:

\"something//\\anotherthing'
\"something//\\anotherthing
\"something/\anotherthing'
\"something\anotherthing'
\\"something\/\/\\\\anotherthing'

When I compare a line read from the file against the pattern in the environment with == operator, I get the expected output:

patt="$pattern" awk '{print $0, ENVIRON["patt"], ($0 == ENVIRON["patt"]?"YES":"NO") }'  OFS="\t" file
\"something//\\anotherthing'    \"something//\\anotherthing'    YES
\"something//\\anotherthing     \"something//\\anotherthing'    NO
\"something/\anotherthing'      \"something//\\anotherthing'    NO
\"something\anotherthing'       \"something//\\anotherthing'    NO
\\"something\/\/\\\\anotherthing'       \"something//\\anotherthing'    NO

But when I do the same with the ~ operator, the tests never match. (I expected YES on the first line, as above):

patt="$pattern" awk '{print $0, ENVIRON["patt"], ($0 ~ ENVIRON["patt"]?"YES":"NO") }'  OFS="\t" file
\"something//\\anotherthing'    \"something//\\anotherthing'    NO
\"something//\\anotherthing     \"something//\\anotherthing'    NO
\"something/\anotherthing'      \"something//\\anotherthing'    NO
\"something\anotherthing'       \"something//\\anotherthing'    NO
\\"something\/\/\\\\anotherthing'       \"something//\\anotherthing'    NO

To fix the issue with ~ comparison I need to double escape the escapes:

patt="${pattern//\\/\\\\}" awk '{print $0, ENVIRON["patt"], ($0 ~ ENVIRON["patt"]?"YES":"NO") }'  OFS="\t" file
\"something//\\anotherthing'    \\"something//\\\\anotherthing' YES
\"something//\\anotherthing     \\"something//\\\\anotherthing' NO
\"something/\anotherthing'      \\"something//\\\\anotherthing' NO
\"something\anotherthing'       \\"something//\\\\anotherthing' NO
\\"something\/\/\\\\anotherthing'       \\"something//\\\\anotherthing' NO

Note the double escapes in result of printing ENVIRON["patt"] in second column.

Question:

Where does escape sequence in awk happening when using tilde ~ comparison operator? on $0 (or $1, $2, ...) or in ENVIRON["variable"]?

6

The ~ operator does pattern matching, treating the right hand operand as an (extended) regular expression, and the left hand one as a string. POSIX says:

A regular expression can be matched against a specific field or string by using one of the two regular expression matching operators, '~' and "!~". These operators shall interpret their right-hand operand as a regular expression and their left-hand operand as a string.

So ENVIRON["patt"] is treated as a regular expression, and needs to have all characters that are special in EREs to be escaped, if you don't want them to be have their regular ERE meanings.


Note that it's not about using $0 or ENVIRON["name"], but the left and right sides of the tilde. This would take the input lines (in $0) as the regular expression to match against:

str=foobar awk 'ENVIRON["str"] ~ $0 { 
     printf "pattern /%s/ matches string \"%s\"\n", $0, ENVIRON["str"] }'
  • Beware that there are several issues in the POSIX specification there which are being addressed at austingroupbugs.net/view.php?id=1105 – Stéphane Chazelas May 23 '18 at 10:49
  • @StéphaneChazelas when it says "Resolution: Accepted As Marked" but "Status: Interpretation Required", what exactly does that mean for the state of this request? – muru May 23 '18 at 11:24
3

A \ in a regular expression escapes the following character, or introduces an escape sequence. To match a literal \ with a regular expression, which is what the ~ operator does in awk, one would need to use \\ (which you do in the last example in the question). In a string comparison, this is not needed.

  • Note that in the case of awk regexp, backslash are also used for escape sequences. With pattern='\b' for instance, it's meant to match on backspace characters (though not all awk implementations do it). There is also some variation between implementations when backslash is used inside bracket expressions. Whether \134 means a litteral backslash also varies with the implementation (as in echo . | P='\134.' awk '$0 ~ ENVIRON["P"]') – Stéphane Chazelas May 23 '18 at 10:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.