awk system call with inverted effect

Question

I have a data file with multiple data blocks enclosed between specific keywords (DATA,END). I am using awk to extract the data blocks into separate files, based on a file name taken from said block. Since some data blocks share the same name, I am renaming each output file with an increasing integer if the file ("blockname") already exists:

#cat input.file
useless stuff1
DATA blockname1
data1
data1
END
useless stuff2
DATA blockname2
data2
data2
END
useless stuff3
DATA blockname1
data3
data3
END
useless stuff4

Expected would be three output files blockname1, blockname2, and blockname1_1 (note how the last file has an integer assigned to it)

#cat blockname1
DATA blockname1
data1
data1
END

(the others accordingly...)

Now the following script works as I want it:

awk '/DATA/,/END/ {
     if ( $1 ~ /DATA/ )
       { block=$2 ; i=0 ; file=block
         while ( system("test ! -e " file ) )
           { i++ ; file=block"_"i ; print file }
       }
       print $0 > file
     }' input.file

My problem lies with the while loop and its system call:

I expected system("test -e " file) to be TRUE when the file exists and to be FALSE if file does not yet exist, i.e. the while loop to only start running if file is present and to break if (the new) file does not exist yet.

However if I use system("test -e " file) (and make it verbose with print file), I have an infinite loop of the same name with increasing integer suffix and the opposite system("test !-e " file) gives the desired result.

So this behaves exactly inverse to what I expected.

Answer 1

OK, I figured: the problem lies with the different definitions of what is TRUE and FALSE between the exit status of test and the while loop condition in awk.

A postive test command results in an exit code of 0 for TRUE and a negative one in 1 for FALSE.

However, in awk the while loop interprets 0 as FALSE and 1 as TRUE, so exactly the opposite definition.

As an example:

awk '{ while ( 0 ) ; { print "0" } }' file

will not produce any output, while

awk '{ while (1) ; { print "1" } }' file

will print infinite 1s.

Best practice is thus to be explicit in such a combination

while ( system("command") == 0 )

or

while ( system("command") == 1 )

respectively.

So in my case

while ( system("test -e " file ) == 0 ) 

shows the expected behaviour.

Answer 2

awk system() returns an exit status of command you run - 0 for success and != 0 if not success. For simple example you can try running:

v = system("date");

v will be 0

if you run:

v = system("dat");

v might be 127 or value different from 0, the error returned from OS if dat command is missing or not found.

Answer 3

If I understand you, the goal is extract the content of input.file into various files avoiding lose blocks with the same name.

If that's the case, and, if the target directory is always empty before the extraction, then there is a better (and faster) solution:

awk '
/DATA/{
    block=$2;
    n = blocks[block]++;
    file=block (n? "_" n: "");  
}
/DATA/,/END/{
    print > file
}' input.file

In this way awk doesn't need execute a new shell N times just to test if file exists.

Notes:

• There's no need the BEGIN block, because awk's field separator is already spaces.
• There's no need the '\' at the end of the lines, because the single quote is already multiline.