6

Reference: The POSIX standard for the awk utility.

One thing I really miss in awk is the ability to join an array with a delimiter, like with the join command in Perl, typically done for immediate output.

Instead, I end up with writing code like

for (key in array)
    joined_string = (joined_string ==  "" ? array[key] : joined_string "," array[key])

print joined_string

or

joined_string = array[1]
for (i = 2; i <= length(array); ++i)
    joined_string = joined_string "," array[i];

print joined_string

However, awk may do this for me if I change the current fields:

OFS="," # (would probably do this in BEGIN)

n = 0
for (key in array)
    $(++n) = array[key]

print

I believe that this is perfectly legal. However, this will produce garbage in the output if the current input record has more fields than the array array has entries (the "garbage" would be data from the input file). It would therefore be nice to be able to do

OFS = "," # (would probably do this in BEGIN)

n = 0
for (key in array)
    $(++n) = array[key]

NF = n
print

I can't find any text in the standard saying that modifying NF is allowed, but neither any text saying that it's not allowed or that it invokes undefined behaviour. The information I can find is that getline sets NF. This does not say that I'm not allowed to write my own function or code block that resets NF, and it gives a precedence for doing this with the existence of the getline "function".

It's also stated that assigning to $0 is allowed and that this resets NF. Does that mean that the following code would be better?

OFS = "," # (would probably do this in BEGIN)

$0 = ""
n = 0
for (key in array)
    $(++n) = array[key]

print

Twofold question:

  1. Is setting NF allowed?
  2. Would that last piece of code be the correct way of joining an array with a delimiter for output?
  • 2
    FWIW, the GNU awk manual has this: "CAUTION: Some versions of awk don’t rebuild $0 when NF is decremented." So, for portability's sake the second might be better. – muru Jun 19 '18 at 11:10
  • @muru Ah, yes, I do recall seeing that, now that you mention it. – Kusalananda Jun 19 '18 at 11:20
  • 1
    @Sundeep no, modifying NF isn’t gawk-specific, at least mawk supports it too. The One True Awk doesn’t. – Stephen Kitt Jun 19 '18 at 11:31
  • 1
    issues with NF assignment is on my long list of issues I plan to raise one day to the austin group. Here, regardless of what POSIX says assigning to NF is not portable in practive. You can always do $0 = "" before though. IIRC length(array) is not standard either. – Stéphane Chazelas Jun 19 '18 at 11:32
  • 1
    see also: stackoverflow.com/questions/42875915/… – Sundeep Jun 19 '18 at 11:34
11

As far as I know there is no standard text documenting the side-effects of setting NF, or even whether setting it is allowed. The Gawk manual (also published as Effective awk Programming), which says that it attempts to document Awk in general and not (only) the GNU implementation, includes the following:

Decrementing NF throws away the values of the fields after the new value of NF and recomputes $0. (d.c.)

with the caveat

CAUTION: Some versions of awk don’t rebuild $0 when NF is decremented.

The “(d.c.)” mention means that this is a “dark corner” of Awk, i.e. one which is poorly documented (or not at all) and where behaviour can vary from one implementation to another.

POSIX defines special variables as

variables that are set by awk

but it doesn’t specify whether they can be set by programs (as a general rule). Some of the variables’ specifications do mention that they can be modified (see ARGC, ARGV), others that the consequences of changing them are implementation-defined (ENVIRON), others still don’t mention anything but are “obviously” intended to be modified by the program (OFS etc.).

In NF’s case, experimentation gives a partial answer:

  • modifying NF works as documented in GNU Awk, and mawk also behaves in the same way;
  • changes to NF in The One True Awk are preserved, but don’t cause $0 to be re-computed.

So I would say that

  1. Setting NF is allowed, but might not have any side-effects other than setting the value.
  2. Since setting $0 is specified by POSIX, the last variant is correct according to the specification. (It’s arguable whether it’s the correct way, since it loses $0.)

The function in How do you convert an array to a string in awk? is interesting, but as defined it relies on GNU Awk extensions and is thus not an answer to this question.

(Other variables which can be set, somewhat surprisingly, include NR and FNR, including in TOTA. FILENAME, however, can’t be set, or rather, setting it clears its value.)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.