I am trying to find and replace the following string:

pete@computer:~$ cat test.txt 

Knowing that the pipe character is a special character I escape it. I also escape the escape character below. My attempt goes here:

pete@computer:~$ awk '{gsub(/~\\|~\\\N$/, "~|~"); print}' test.txt 

The expected result is:


but I'm getting additional "|~" in there. What am I doing wrong?

Thanks, Pete

  • 1
    Why not replace only \N with nothing? sed 's/\\N//g'
    – Panki
    Jun 24, 2021 at 10:32
  • @Panki I'm assuming this is data that is later fed into a database which uses \N to mean NULL (~ is used as an quoting character, and | is the field delimiter). Removing all \N may be the wrong thing to do as some fields possibly should be \N. However, the suggested approach in the question, to match the field delimiter before the \N would in that case also break the data. To properly solve this, the error in the code that produces the data should be fixed.
    – Kusalananda
    Jun 24, 2021 at 10:52
  • @Kusalananda thanks, learned something new today. Guess this should have been stated in the OP.
    – Panki
    Jun 24, 2021 at 11:09
  • You started out right by identifying that | and \ are regexp meta chars that need to be escaped but then instead of just escaping them as \| and \\ you chose to add a second set of backslashes turning it into \\| and \\\ which is no longer escaping the original | and \ but is now escaping the \ you added to escape those chars and so unescaping them and thereby making them metachars again. You then added a $ to the end of your regexp so the part after the | could now only match at the end of the line when you wanted it to match everywhere on the line.
    – Ed Morton
    Jun 24, 2021 at 11:44
  • If you tell us what made you think you had to add those extra \s and $ we could help explain the flaw in that logic to help you avoid similar issues in future.
    – Ed Morton
    Jun 24, 2021 at 11:48

1 Answer 1

awk '{ gsub(/~\|~\\N/, "~|~") }1' infile

single escape is needed for special characters in regex argument to the sub()/gsub()/gensub() functions and also you would need to remove the $ that is end-of-match anchor.

when you used /~\\|~\\\N$/, it matches two regexes ~\ or ~\N but second matches only if it was at the end of the line which $ telling; so according to these you have only ~\ that can match on your input based on the regex and that ~\ is replacing with ~|~ and produce below output:


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .