1

For a file containing records such as:

ABC|YE0000123543|BLAH|XYZ|24.12.2025|J
ABC|YE0000123544|BLAH|MNO|24.12.2025|K

If the fourth field contains XYZ, replace it with a Y. If not, replace it with N.

Sounds simple, right? Well apparently not matching something with awk is extremely difficult, if not impossible, or I'm just not getting it.

Substitute for a match:

$ echo "ABC|YE0000123543|BLAH|XYZ|24.12.2025|J"|awk -F '|' 'BEGIN { OFS=FS } {gsub(/XYZ/, "Y", $4);} {print $0}'
ABC|YE0000123543|BLAH|Y|24.12.2025|J

So far so good! Now to print N for a non-match:

Just negate the regex, right? !/XYZ/

$ echo "ABC|YE0000123543|BLAH|MNO|24.12.2025|J"|awk -F '|' 'BEGIN { OFS=FS } {gsub(!/XYZ/, "N", $4);} {print $0}'
ABC|YE0000123543|BLAH|MNO|24.12.2025|J

No? It just doesn't work.

Looking at other answers on SO leaves me none the wiser. Nothing seems to be able to reliably substitute for a non-matching regex in awk.

Any pointers would be appreciated.

3
  • ITYM If the fourth field is XYZ rather than If the fourth field contains XYZ. Regarding apparently not matching something with awk is extremely difficult, - using any tool it's almost always trivial to match whatever you want but much harder to not match similar text you don't want.
    – Ed Morton
    Commented Jul 12, 2022 at 12:13
  • Regarding Just negate the regex, right? !/XYZ/ - no, there is no negation of regexps (in BREs or EREs anyway, idk about PCREs). You can negate a condition that's testing a regexp but that's not negating the regexp. You can negate a bracket expression within a regexp, e.g. [!a] would match any character that isn't a but again that's not negating a whole regexp. sub(/XYZ/ means "replace XYZ", but sub(!/XYZ/ means "replace 0 if XYZ exists in $0 or 1 otherwise".
    – Ed Morton
    Commented Jul 12, 2022 at 12:26
  • To "negate a regexp" like XYZ you'd first have to figure out what that means. Given input like abcXYZ, obviously XYZ DOES match XYZ but what's the negation of that? Is it abc? Or is is a, and b, and c individually? Does it include the null strings between each character? Does it match XY and YZ and bcX, etc.? There's no obvious answer. What you probably had in mind when expressed as a regexp would be written as sub(/^([^X]|.[^Y]|..[^Z]|.{4,})$/ but it's not exactly elegant and it's not as simple as just negating the original regexp using some negation operator.
    – Ed Morton
    Commented Jul 12, 2022 at 12:40

5 Answers 5

4

As @Romeo also suggested, string equality can be more accurate than a regex.

I'd write

awk 'BEGIN {FS=OFS="|"} {$4 = ($4 == "XYZ" ? "Y" : "N")} 1' file
1
  • This is perfect. Thanks very much man. So obvious it's embarassing. Commented Jul 10, 2022 at 18:06
2

You cat change script like:

$ awk -F '|' 'BEGIN { OFS=FS } {if("XYZ"==$4) $4="Y" ;else $4="N"; print }' input_file >output_file
0

There are umpteen ways to skin a cat... You seem to want to replace the entire field 4, no? Try a "conditional" operator:

awk -F '|' '{$4=($4=="XYZ")?"Y":"N"} 1' OFS="|" file5
ABC|YE0000123543|BLAH|Y|24.12.2025|J
ABC|YE0000123544|BLAH|N|24.12.2025|K
1
  • Sorry but @glennjackman beat you to it (posted the exact same answer, somewhat earlier). And it is a community answer (if you want to contribute). Commented Jul 10, 2022 at 18:05
0

Your original code contains a bare regular expression, which is matched against the whole line. That may cause unintended matches.

If you really need a pattern match (not just a text equality), the syntax to match a field with a pattern is like:

$4 ~ /myRegExp/

That can be negated with an operator like:

$4 !~ /myRegExp/

The sub() function returns the number of substitutions, so you can force alternatives like:

if (! sub (/myRegExp/, "Text_A", $4)) $4 = "myDefault";

For free-format text (rather than fields with FS), the match() function can be very helpful. It returns built-in variables RSTART and RLENGTH, which permit you to use substr() to reference the matched text, rather than the field notation like $4.

0

Using Raku (formerly known as Perl_6)

~$ raku -ne 'my @a = .split("|"); @a.[3] eq "XYZ" ?? (@a.[3] = "Y") !! (@a.[3] = "N"); @a.join("|").put;'  file

OR

~$ raku -ne 'my @a = .split("|"); @a.[3] = (@a.[3] eq "XYZ" ?? "Y" !! "N"); @a.join("|").put;'  file

Does it have to be awk? Here's a solution using Raku's ternary operator: "(Condition) ?? True !! False". The nice thing about the split/join approach above is it sets you up if you want to join on commas, to make a simple-csv file. Otherwise, the Raku reads similar to the awk answers posted, noting that Raku (and Perl5) are zero-indexed.

https://docs.raku.org/language/operators#index-entry-operator_ternary
https://raku.org

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .