4

I have always been using single-quotes for the field separation like: awk -F';' ...

Quite new to me is the way of using a backslash like: awk -F\; ...

is there a technical difference for either, or is it just a matter of preference?

8

That's all to do with your shell, not with awk.

In Bourne-like shells, \, '...' and "..." are all quoting operators.

Quoting removes the special meaning a character may have in the syntax of the shell. \ quotes a single character (except for newline which it removes instead), '...' and "..." can quote more than one (with "..." not quoting every character).

; is a special character in the syntax of the shell. It's used to separate commands. You want to quote it if you want to pass it verbatim to a command. \;, ';' will do.

";" will also do as ; is not one of those characters that are still special within double quotes, but you'd need "\\" to pass one literal backslash to a command because \ is one of those characters that are still special within "..." (though it's then only special when followed by other special characters within "..." like that " itself).

Again that very much depends on the shell. In the rc shell for instance, \ and " are not special let alone quoting characters, -F\; wouldn't work there as the command would be parsed as both the awk -F\ and ... command separated with ;.

See How to use a special character as a normal one? for more details.

To complicates things further, note that the argument to -F itself also goes through one or two layers of backslash processing by awk.

awk processes first the argument it receives to expand ANSI C escape sequences in it. If you use awk -F '\t' or awk -F \\t or awk -F "\\t" or awk -F "\t", awk receives an argument that contains \t, which it expands to a TAB character. The FS awk variable will contain a TAB character, not \t.

With awk -F '\\', awk receives a \\ argument and sets FS to the \ character. Strictly speaking, awk -F '\' would is unspecified as that escape sequence is unfinished but in practice, except for busybox awk, all awk implementations I know treat it the same as awk -F '\\'.

In awk, when FS contains a single character, that character is the field separator. awk -F . splits the records on dot characters.

However when FS contains more than one character, it is interpreted as a regular expression. awk -F .. doesn't spilt on sequences of two dots, but on sequences of any two characters as . is the regular expression operator that matches any single character. To split on two dots, you'd need awk -F '[.][.]' or awk -F '\\.\\.'.

With awk -F '\\\\', a literal \\\\ is passed by the shell to awk, awk expands each of those two \\ to \, so FS becomes \\, which is treated as a regular expression. \ is also special in the regular expression syntax and is used to remove the special meaning of a character as a regex operator this time. So again, that is splitting on backslash characters, though this time, as a regular expression.

So, in practice, to split on \, all of these (in Bourne-like shells) will work:

awk -F '\'      # FS becomes a single \ except in busybox where it's empty
awk -F "\\"     # instead so it's a one-character split on backslash
awk -F \\       # and a one-field-by-character split in busybox

awk -F '\\'     # FS becomes a single \ in every awk implementation
awk -F \\\\     # so one-character split on backslash
awk -F "\\\\"

awk -F '\\\'    # FS is \ on busybox and \\ in other implementations
awk -F \\\\\\   # so one-character split on backslash in busybox and
awk -F "\\\\\\" # \\ regex split in other implementations, to the same effect

awk -F '\\\\'     # FS is \\ in all implementations so
awk -F \\\\\\\\   # \\ regex split
awk -F "\\\\\\\"

I would advise to use single quotes as they are the most straightforward and least surprising kind of quotes. So here, to split on backslash portably: awk -F '\\'.

You can also do things like:

 awk -v FS='\\' ...

Or

 awk 'BEGIN{FS="\\"} ...'

or

awk ... 'FS=\\'

or:

FS='\' awk 'BEGIN{FS = ENVIRON["FS"]} ...'

(that one avoiding the extra backslash expansion performed by awk, so need only one backslash).

4

All characters within single quotes are treated literally (i.e. no character is special between a pair of single quotes). Without single quotes, you need to backslash-escape a character with a special meaning if you want to use the literal character.

These are the quoting rules of the shell and are unrelated to awk.

  • so if the field separator would not be a special character lets say it was a , one could also just use a command like: awk -F, ...? – nath May 25 at 4:31
  • 2
    Yes, that's right. – Niko Gambt May 25 at 4:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.