I have a data file with multiple data blocks enclosed between specific keywords (DATA,END). I am using awk to extract the data blocks into separate files, based on a file name taken from said block. Since some data blocks share the same name, I am renaming each output file with an increasing integer if the file ("blockname") already exists:

#cat input.file
useless stuff1
DATA blockname1
useless stuff2
DATA blockname2
useless stuff3
DATA blockname1
useless stuff4

Expected would be three output files blockname1, blockname2, and blockname1_1 (note how the last file has an integer assigned to it)

#cat blockname1
DATA blockname1

(the others accordingly...)

Now the following script works as I want it:

awk '/DATA/,/END/ {
     if ( $1 ~ /DATA/ )
       { block=$2 ; i=0 ; file=block
         while ( system("test ! -e " file ) )
           { i++ ; file=block"_"i ; print file }
       print $0 > file
     }' input.file

My problem lies with the while loop and its system call:

I expected system("test -e " file) to be TRUE when the file exists and to be FALSE if file does not yet exist, i.e. the while loop to only start running if file is present and to break if (the new) file does not exist yet.

However if I use system("test -e " file) (and make it verbose with print file), I have an infinite loop of the same name with increasing integer suffix and the opposite system("test !-e " file) gives the desired result.

So this behaves exactly inverse to what I expected.

3 Answers 3


OK, I figured: the problem lies with the different definitions of what is TRUE and FALSE between the exit status of test and the while loop condition in awk.

A postive test command results in an exit code of 0 for TRUE and a negative one in 1 for FALSE.

However, in awk the while loop interprets 0 as FALSE and 1 as TRUE, so exactly the opposite definition.

As an example:

awk '{ while ( 0 ) ; { print "0" } }' file

will not produce any output, while

awk '{ while (1) ; { print "1" } }' file

will print infinite 1s.

Best practice is thus to be explicit in such a combination

while ( system("command") == 0 )


while ( system("command") == 1 )


So in my case

while ( system("test -e " file ) == 0 ) 

shows the expected behaviour.


awk system() returns an exit status of command you run - 0 for success and != 0 if not success. For simple example you can try running:

v = system("date");

v will be 0

if you run:

v = system("dat");

v might be 127 or value different from 0, the error returned from OS if dat command is missing or not found.


If I understand you, the goal is extract the content of input.file into various files avoiding lose blocks with the same name.

If that's the case, and, if the target directory is always empty before the extraction, then there is a better (and faster) solution:

awk '
    n = blocks[block]++;
    file=block (n? "_" n: "");  
    print > file
}' input.file

In this way awk doesn't need execute a new shell N times just to test if file exists.


  • There's no need the BEGIN block, because awk's field separator is already spaces.
  • There's no need the '\' at the end of the lines, because the single quote is already multiline.
  • Well, my problem was more in understanding the "odd" behaviour of the while condition. However your solution is pretty neat - thank you, I prefer, this over my bulky thing. The BEGIN block is a leftover from my different file format - I forgot it becomes useless in my generic example. Leaving out the backslash will spare me some trouble. Thanks for this, too. But you are spot on regarding the script's intention. Could you explain the n = blocks[block]++; file=block (n? "_" n: "") part in some more detail?
    – FelixJN
    Nov 16, 2016 at 19:40
  • 1
    The algorithm uses one array: blocks that is indexed by block name. Ex. In the first instance of "blockname1": blocks["blockname1"]. Awk find that index and because it isn't found assumes "" (also considered zero). Now, in awk n = var++ is equivalent to {n=var;var++}, so n=="" and blocks["blockname1"]==1. Finally file=block (n? "_" n: "") is the same as {file=block;if(n!="") file+="_" n}. Nov 18, 2016 at 2:33
  • I wasn't even aware a string is a valid indexing variable and of is this reduced if-statement - very helpful, thanks again. I'm saddened I cannot give you the "accepted answer" here: despite me going to adopt this approach as it fulfils my initial task, it doesn't answer the question above (interpretation of condition in while loop) - it would go against my understanding of this sites Q&A system. Yet, I learned a lot - thanks again.
    – FelixJN
    Nov 21, 2016 at 10:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.