-3

I am trying to figure out how to write a standalone awk script file.

I thought it would be similar to a standalone bash script file:

#! /usr/bin/awk -f      
BEGIN{
    for  (i  =  0;  i  <  ARGC;  i++)
        printf  "%s  ",  ARGV[i]
    printf  "\n"
}
{print $0}
  1. I was trying to figure out how the command line arguments are specified in shell, and passed into the script:

    $ myscript.awk arg1 arg2 arg3
    awk  arg1  arg2  arg3  
    awk: /home/tim/myscript.awk:5: fatal: cannot open file `arg1' for reading (No such file or directory)
    

    What does an awk script expect its command line arguments to be? Why does it expect arg1 to be the input file?

    Command line arguments are passed into an awk script, and stored in array ARGV. See my udpate. So I suppose the command line arguments are interpreted up to the script, not toawk.

  2. If I remove -f in the shebang, i.e. #! /usr/bin/awk

    $ myscript.awk arg1 arg2 arg3
    awk: cmd. line:1: /home/tim/myscript.awk
    awk: cmd. line:1:                   ^ syntax error
    

    Why is -f necessary?

Thanks.

  • What do you expect arg1 to be (or want it to be)? awk always takes the target filenames as arguments. – Michael Homer Aug 20 '17 at 6:22
  • command line arguments are passed into an awk script, and stored in array ARGV. See my udpate. So I suppose the command line arguments are interpreted up to the script, not to awk. – Tim Aug 20 '17 at 6:30
  • And {print $0} is expected to be operating on...? – Michael Homer Aug 20 '17 at 6:36
  • @Michael So input files must be specified, and must be specified as command line arguments. Are all the command line arguments expected to be input filenames? Can there be some command line arguments which are not input files but are interpreted up to the script? – Tim Aug 20 '17 at 6:38
3

What does an awk script expect its command line arguments to be? Why does it expect arg1 to be the input file?

awk's pattern based rules need input. When processing of this parts of your program starts, awk starts to consume arguments as filenames (or stdin if no filenames are given).

Before this step you can do whatever you want with given argments in the BEGIN block.

I think, these small examples get you started:

$ cat a.awk 
#!/usr/bin/awk -f
BEGIN {
        i=1
        while( i in ARGV )
                print ARGV[i++]
}

a.awk only has a BEGIN block and no pattern based rules. awk does not need files and so does not use the given arguments as filenames:

$ ./a.awk poit --zort -troz narf
poit
--zort
-troz
narf

It is your decision what to do with these.

If you want to have pattern based rules processing files given as arguments too, you need to delete all arguments you have used in your BEGIN block:

$ cat b.awk 
#!/usr/bin/awk -f
BEGIN {
        if( ARGV[1] == "--tolower" ) { cmd = "tr A-Z a-z" ; delete ARGV[1] }
        else if( ARGV[1] == "--toupper" ) { cmd = "tr a-z A-Z" ; delete ARGV[1] }
        else cmd = "cat"
}
{
        print | cmd
}

Example run without option:

$ ./b.awk a.awk
#!/usr/bin/awk -f
BEGIN {
        i=1
        while( i in ARGV )
                print ARGV[i++]
}

Example run with --toupper option:

$ ./b.awk --toupper a.awk
#!/USR/BIN/AWK -F
BEGIN {
        I=1
        WHILE( I IN ARGV )
                PRINT ARGV[I++]
}
  • Thanks. I wonder what you meant by "if stdin already is fully used up"? – Tim Aug 20 '17 at 22:20
  • awk can use stdin as input but I mixed something up. It uses stdin (like in echo 'Hi!' | awk '{ print}') only if there are no filenames given as arguments. man mawk writes: "If ARGC equals 1, the input stream is set to stdin, else the command line arguments ARGV[1] ... ARGV[ARGC-1] are examined for a file argument." – yeti Aug 21 '17 at 6:30
  • AAAahhhhh! Now I got it again... \o/ ...yesterday I asked myself why I have put the whole ncomm only in a BEGIN block. Some of its functionality looks like just right for pattern based rules. But: Bingo! Pattern based rules would allow stdin as input which would make no sense here because it only would be the only "file" and even would not give me a filename for the header of the output. Well... no program should be too short for some comments or documentation... I'll let awk write this 1000 times for me... ;-) – yeti Aug 21 '17 at 7:05
  • With some awk implementations, that --toupper would be taken as an option to awk, not the awk script, so portably, you'd need to invoke your script as ./b.awk -- --toupper a.awk, -- marking the end-of-options for awk, so the --toupper be passed as an argument to the script. In any awk implementation, you'll have a problem with options that happen to be valid awk options. – Stéphane Chazelas Aug 21 '17 at 9:42
  • (a) That's really mean! Which behaviour is defined for this in the original awk? (b) That does not change that you need to delete every ARGV[*] your BEGIN block used if you don't want it to be reused as filename for pattern rules. So if that turns out to be a major problem, maybe the "delete used args example" should not show something related to options... – yeti Aug 21 '17 at 10:12
6

AWK expects its arguments to be either a script’s text, or -f followed by the name of a file containing the script to run, in both cases optionally followed by the names of the files to process.

This explains why you need -f in the shebang line: without that, AWK considers that your script’s filename is itself the AWK statements to run.

Regarding argument processing, an AWK program can do its own argument processing if it wishes. Your script fails because of the {print $0} line: this instructs the interpreter to read each line from its input (the files named on the command line, since you’ve specified some) and process it according to the instructions in the block. If you remove that line you won’t get any errors. You can process arguments in BEGIN and clean up ARGV so that it only contains input files; then AWK won’t complain.

Trying to do your own argument and input processing entirely, would probably mean ignoring a lot of what makes AWK so useful; if you want to do that you might as well use Perl.

(Note that the shebang handling means you can store simple AWK scripts in their filename, which avoids having to find clever names for scripts — not that anyone should do that...)

  • Thanks. Is the syntax of Perl more similar to awk or to sed? – Tim Aug 20 '17 at 18:38
  • Thanks. I was wondering what you mean by " shebang handling means you can store simple AWK scripts in their filename, which avoids having to find clever names for scripts — not that anyone should do that..."? Thanks. – Tim Aug 20 '17 at 20:07
  • Perl is much closer to AWK than to sed. My note was perhaps a too-clever comment pointing out that you can write a script called { print NF } containing only #!/usr/bin/awk, and it will work (if it’s on your path). That’s not specific to AWK of course, it works with any interpreter capable of interpreting commands given as arguments to it (including the shell, with -c). – Stephen Kitt Aug 20 '17 at 20:36
  • A problem with #! /usr/bin/awk -f is that if you pass options (arguments starting with -, though that's limited to options recognised by awk with some awk implementations) to the script, they will be interpreted as awk options (not arguments to the script). There are also potential problems with arguments containing = characters. See #! /usr/bin/gawk -E to work around those. – Stéphane Chazelas Aug 21 '17 at 9:43
  • @StéphaneChazelas To distinguish awk options and arguments to the script, is it a way to process those elements in ARGV that are meant for the script in BEGIN block? – Tim Aug 21 '17 at 12:16
4

An awk script expects its non-option command line arguments to be filenames of files upon which the script should act (if none are given, it acts on standard input).

So when you use #!/usr/bin/awk -f in an awk script file, this tells the system that the text of the file itself should be passed on to awk -f. Any other command line arguments will be interpreted as input files or as additional flags to awk:

#!/usr/bin/awk -f

BEGIN {
    for (i in ARGV) {
        printf("ARGV[%d] = %s\n", i, ARGV[i]);
    }
    printf("var = %s\n", var);
}

$ ./script.awk -vvar=hello ~/.profile
ARGV[0] = awk
ARGV[1] = /home/kk/.profile
var = hello

Command line checks for options ends at the first non option argument:

$ ./script.awk ~/.profile -vvar=hello
ARGV[2] = -vvar=hello
ARGV[0] = awk
ARGV[1] = /home/kk/.profile
var =

For awk to be useful for anything, it needs input data. This data usually comes from one or several input files named on the command line, or from sending data on the standard input stream.

Apart from the BEGIN and END blocks, each block in an awk script will be applied to each record (each line by default) of the input data in turn.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.