9

In awk, the first argument to the sub() and gsub() functions, the second argument to the match() function, and the optional third argument to split(), is an extended regular expression.

Such an argument may be an arbitrary expression that evaluates to a string that is interpreted as a regular expression, or it may be a regular expression constant.

In awk, a regular expression constant is written /RE/ for some regular expression RE (an "ERE token" in the awk grammar).

Question: Assuming that RE remains the same (some non-variable regular expression), are there any practical differences between using /RE/ and using "RE" (a "STRING token" in the awk grammar) in a call to, e.g., sub()? Alternatively: Is there any known awk implementation that makes a difference between these two ways of representing a regular expression in a call to the above-mentioned functions?

The reason for asking is that I recall having to modify some awk code that tried to use "RE" as a regular expression in a call to either sub() or gsub(), because, for whatever reason, the awk implementation at hand did the wrong thing unless I called the function with /RE/.

Unfortunately, this was some time ago (2+ years) so I don't remember the details and I am even uncertain what Unix I used at the time (possibly OpenBSD), but I've been meaning to ask the question ever since.

9
  • 1
    Related: austingroupbugs.net/view.php?id=1105 Commented Jan 17, 2023 at 14:59
  • @StéphaneChazelas That's certainly a relevant text.
    – Kusalananda
    Commented Jan 17, 2023 at 15:01
  • IIRC, there were several discussions around that on the austin group mailing list around that time. My vague recollection was that there was quite a bit of variations between implementations. IIRC, the POSIX position was that $0 ~ "\f" matches against a re that contains a formfeed character, while $0 ~ /\f/ matches against a re that contains \f, but that's meant to match a form feed for the regexp engine. I don't remember what /[\f]/ was meant to match (whether a FF or either backslash or f) Commented Jan 17, 2023 at 15:05
  • In any case, you need to double your backslashes inside "..." so it becomes a RE escaping operator (but see above about \f/\b...) and quadruple them ($0 ~ "\\\\") to match a literal backslash. Commented Jan 17, 2023 at 15:06
  • With busybox awk, you can use $0 ~ "(.)\\1" for back references, but IIRC, that's not POSIX compliant as POSIX requires that to match the same as $0 ~ /(.)\1/, so any character followed by byte 0x1 Commented Jan 17, 2023 at 15:08

1 Answer 1

11

Yes, they are different in all awk implementations in the way that backslashes are used as escapes. See https://www.gnu.org/software/gawk/manual/gawk.html#Computed-Regexps for this and more details:

... be aware that there is a difference between a regexp constant enclosed in slashes and a string constant enclosed in double quotes. If you are going to use a string constant, you have to understand that the string is, in essence, scanned twice: the first time when awk reads your program, and the second time when it goes to match the string on the lefthand side of the operator with the pattern on the right. This is true of any string-valued expression (such as digits_regexp, shown in the previous example), not just string constants.

What difference does it make if the string is scanned twice? The answer has to do with escape sequences, and particularly with backslashes. To get a backslash into a regular expression inside a string, you have to type two backslashes.

For example, /\*/ is a regexp constant for a literal *. Only one backslash is needed. To do the same thing with a string, you have to type "\\*". The first backslash escapes the second one so that the string actually contains the two characters \ and *.

So, when writing a regexp the only time to consider using "RE" instead of /RE/ is when:

  1. The RE contains /s and no escapes so you can write, for example, sub("/foo/bar","") instead of sub(/\/foo\/bar/,"") just for conciseness, or
  2. You NEED string delimiters for concatenation like var=17; sub("stuff" var,"")

otherwise always use regexp delimters (/) around regexps, not string delimiters (").

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .