How to write an awk script?

Question

I am trying to figure out how to write a standalone awk script file.

I thought it would be similar to a standalone bash script file:

#! /usr/bin/awk -f      
BEGIN{
    for  (i  =  0;  i  <  ARGC;  i++)
        printf  "%s  ",  ARGV[i]
    printf  "\n"
}
{print $0}

1. I was trying to figure out how the command line arguments are specified in shell, and passed into the script:

   $ myscript.awk arg1 arg2 arg3
   awk  arg1  arg2  arg3  
   awk: /home/tim/myscript.awk:5: fatal: cannot open file `arg1' for reading (No such file or directory)
   

   What does an awk script expect its command line arguments to be? Why does it expect arg1 to be the input file?

   Command line arguments are passed into an awk script, and stored in array ARGV. See my udpate. So I suppose the command line arguments are interpreted up to the script, not toawk.

2. If I remove -f in the shebang, i.e. #! /usr/bin/awk

   $ myscript.awk arg1 arg2 arg3
   awk: cmd. line:1: /home/tim/myscript.awk
   awk: cmd. line:1:                   ^ syntax error
   

   Why is -f necessary?

Thanks.

Answer 1

AWK expects its arguments to be either a script’s text, or -f followed by the name of a file containing the script to run, in both cases optionally followed by the names of the files to process.

This explains why you need -f in the shebang line: without that, AWK considers that your script’s filename is itself the AWK statements to run.

Regarding argument processing, an AWK program can do its own argument processing if it wishes. Your script fails because of the {print $0} line: this instructs the interpreter to read each line from its input (the files named on the command line, since you’ve specified some) and process it according to the instructions in the block. If you remove that line you won’t get any errors. You can process arguments in BEGIN and clean up ARGV so that it only contains input files; then AWK won’t complain.

Trying to do your own argument and input processing entirely, would probably mean ignoring a lot of what makes AWK so useful; if you want to do that you might as well use Perl.

(Note that the shebang handling means you can store simple AWK scripts in their filename, which avoids having to find clever names for scripts — not that anyone should do that...)

Answer 2

What does an awk script expect its command line arguments to be? Why does it expect arg1 to be the input file?

awk's pattern based rules need input. When processing of this parts of your program starts, awk starts to consume arguments as filenames (or stdin if no filenames are given).

Before this step you can do whatever you want with given argments in the BEGIN block.

I think, these small examples get you started:

$ cat a.awk 
#!/usr/bin/awk -f
BEGIN {
        i=1
        while( i in ARGV )
                print ARGV[i++]
}

a.awk only has a BEGIN block and no pattern based rules. awk does not need files and so does not use the given arguments as filenames:

$ ./a.awk poit --zort -troz narf
poit
--zort
-troz
narf

It is your decision what to do with these.

If you want to have pattern based rules processing files given as arguments too, you need to delete all arguments you have used in your BEGIN block:

$ cat b.awk 
#!/usr/bin/awk -f
BEGIN {
        if( ARGV[1] == "--tolower" ) { cmd = "tr A-Z a-z" ; delete ARGV[1] }
        else if( ARGV[1] == "--toupper" ) { cmd = "tr a-z A-Z" ; delete ARGV[1] }
        else cmd = "cat"
}
{
        print | cmd
}

Example run without option:

$ ./b.awk a.awk
#!/usr/bin/awk -f
BEGIN {
        i=1
        while( i in ARGV )
                print ARGV[i++]
}

Example run with --toupper option:

$ ./b.awk --toupper a.awk
#!/USR/BIN/AWK -F
BEGIN {
        I=1
        WHILE( I IN ARGV )
                PRINT ARGV[I++]
}

Answer 3

An awk script expects its non-option command line arguments to be filenames of files upon which the script should act (if none are given, it acts on standard input).

So when you use #!/usr/bin/awk -f in an awk script file, this tells the system that the text of the file itself should be passed on to awk -f. Any other command line arguments will be interpreted as input files or as additional flags to awk:

#!/usr/bin/awk -f

BEGIN {
    for (i in ARGV) {
        printf("ARGV[%d] = %s\n", i, ARGV[i]);
    }
    printf("var = %s\n", var);
}

$ ./script.awk -vvar=hello ~/.profile
ARGV[0] = awk
ARGV[1] = /home/kk/.profile
var = hello

Command line checks for options ends at the first non option argument:

$ ./script.awk ~/.profile -vvar=hello
ARGV[2] = -vvar=hello
ARGV[0] = awk
ARGV[1] = /home/kk/.profile
var =

For awk to be useful for anything, it needs input data. This data usually comes from one or several input files named on the command line, or from sending data on the standard input stream.

Apart from the BEGIN and END blocks, each block in an awk script will be applied to each record (each line by default) of the input data in turn.