3

I got a test file, its content is:

a -*- b

I used awk 'BEGIN {FS="*"} {print $2}' test, it prints out

- b

Correct! But when I use awk 'BEGIN {FS="-*-"} {print $2}' test, I got:

*

I know that FS support regex, so I added \ before *, I did awk 'BEGIN {FS="-\*-"} {print $2}' test , still, I got:

*

Luckily, I got a blog write by myself half a year ago. Which mentioned I should use awk 'BEGIN {FS="-[*]-"} {print $2}' test in this case. Thus I got:

 b

Correct again!

But I was really confusing why FS can understand *, can't understand -*- and -\*-, and finally can understand the -[*]-.

What's the mechanism in it?

4

If FS is longer than a single character, it is treated as a regular expression. An FS of just * is seen as a fixed string, but an FS of -*- is a regular expression, and -*- is equivalent to -+ (one or more -). So you need to make * be considered as a regular character. -\*- and -[*]- can both do this. However, the string for FS is parsed twice - once when you're assigning it, and once for splitting on FS. That's why \-escaped characters need to have the \ escaped as well.

$ awk -F '-\\*-' '{print $2,FS}' test.txt
 b -\*-
$ awk -F '-\*-' '{print $2,FS}' test.txt
awk: warning: escape sequence `\*' treated as plain `*'
* -*-
  • Why -*- is equivalent to -+-? – cuonglm Feb 6 '15 at 4:52
  • @cuonglm: It isn't, but that's not what muru said. He said that -*- is equivalent to -+ but then he followed that with a '-' outside the backticks, which lead to your confusion. – PM 2Ring Feb 6 '15 at 4:55
  • So FS='ax\*' and FS='ax*' has no difference? – Zen Feb 6 '15 at 4:57
  • @PM2Ring: The same question, why -*- is equivalent to -+? – cuonglm Feb 6 '15 at 4:57
  • @cuonglm -* means 0 or any -, - means one fixed -. -*- means 0 or any - plus one fixed -. -+ means one or any -. They are different in order to create the -. But as all - are identical to us, these two command are equivalent. – Zen Feb 6 '15 at 5:01
3

A key point in muru's answer is that to get a backslash into the FS regex you need to write double backslash \\. That's because backslash is used as an escape character at two different levels.

A single backslash in a string will be treated as escaping the following character, so we need to escape the backslash itself so that we get a single backslash in the regex. And then that backslash will escape the following character within the regex.

As I said in a comment, there's no difference between FS='ax\*' and FS='ax*' because \* is treated as *, but awk will print a warning to that effect. If you want to put a literal * into the FS you need to use double backslash, eg FS='ax\\*' will split on ax*.

Maybe some examples will make all this a bit clearer.

#!/usr/bin/env bash

s='123abcd
123axbcd
123axxbcd
123ax*bcd
123ax**bcd'

printf "%s\n\n" "$s"

awk -F 'ax*' 'BEGIN{printf "FS=[%s]\n", FS};{printf "[%s] [%s]\n", $1, $2}' <<< "$s"
echo

awk 'BEGIN{FS="ax*"; printf "FS=[%s]\n", FS};{printf "[%s] [%s]\n", $1, $2}' <<< "$s"
echo


awk -F 'ax\*' 'BEGIN{printf "FS=[%s]\n", FS};{printf "[%s] [%s]\n", $1, $2}' <<< "$s"
echo

awk 'BEGIN{FS="ax\*"; printf "FS=[%s]\n", FS};{printf "[%s] [%s]\n", $1, $2}' <<< "$s"
echo


awk -F 'ax\\*' 'BEGIN{printf "FS=[%s]\n", FS};{printf "[%s] [%s]\n", $1, $2}' <<< "$s"
echo

awk 'BEGIN{FS="ax\\*"; printf "FS=[%s]\n", FS};{printf "[%s] [%s]\n", $1, $2}' <<< "$s"
echo

output

123abcd
123axbcd
123axxbcd
123ax*bcd
123ax**bcd

FS=[ax*]
[123] [bcd]
[123] [bcd]
[123] [bcd]
[123] [*bcd]
[123] [**bcd]

FS=[ax*]
[123] [bcd]
[123] [bcd]
[123] [bcd]
[123] [*bcd]
[123] [**bcd]

awk: warning: escape sequence `\*' treated as plain `*'
FS=[ax*]
[123] [bcd]
[123] [bcd]
[123] [bcd]
[123] [*bcd]
[123] [**bcd]

awk: warning: escape sequence `\*' treated as plain `*'
FS=[ax*]
[123] [bcd]
[123] [bcd]
[123] [bcd]
[123] [*bcd]
[123] [**bcd]

FS=[ax\*]
[123abcd] []
[123axbcd] []
[123axxbcd] []
[123] [bcd]
[123] [*bcd]

FS=[ax\*]
[123abcd] []
[123axbcd] []
[123axxbcd] []
[123] [bcd]
[123] [*bcd]
1

Within the " delimiter, you need to escape the backslash one mre time.

$ echo 'a -*- b' | awk 'BEGIN {FS="-\\*-"} {print $2}'
 b

Since we are passing a regex to the FS variable, \\ within the double quotes is parsed as single backslash and then it apply the resultant regex against the input string.

  • What's the logic of escape two times? – Zen Feb 6 '15 at 4:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.