BEGIN and END with the awk command

Question

According to the awk manual, BEGIN and END are not used to match input, but rather to provide start-up and clean-up information to the awk script. Here is the example given:

ls -l | \
awk 'BEGIN { print "Files found:\n" } /\<[a|x].*\.conf$/ { print $9 }'
Files found:
amd.conf
antivir.conf
xcdroast.conf
xinetd.conf

First this prints a string to output. Then it checks input for a pattern match, where the input starts with a or x followed by any character one or many times followed by the .conf. For any matches, the 9th column is printed.

The fact that we are forced to use begin here, does that mean awk can only use at most one print function that does contain a BEGIN or END? If not, then why can't we just use the print function at the beginning without the keyword BEGIN? It seems the BEGIN is superfluous.

Answer 1

The BEGIN isn't superfluous. If you don't specify BEGIN then the print would be executed for every line of input.

Quoting from the manual:

A BEGIN rule is executed once only, before the first input record is read. Likewise, an END rule is executed once only, after all the input is read.

$ seq 5 | awk 'BEGIN{print "Hello"}/4/{print}'   # Hello printed once
Hello
4
$ seq 5 | awk '{print "Hello"}/4/{print}'        # Hello printed for each line of input
Hello
Hello
Hello
Hello
4
Hello
$

Answer 2

awk processes each line of input for the expressions given in the body other than BEGIN and END blocks. In the case of BEGIN and END blocks, awk will process the statements only once, before the processing of input has begun and after the processing of input has been done respectively. Without the BEGIN block, not only you would not be able to print one-time information such as headers, you would not be able to efficiently initialize some of the variables required by the body. Also, FYI, an awk program can have multiple BEGIN and END blocks.

Answer 3

awk executes every block only when the pattern before it matches. Empty pattern (just block) matches every line. BEGIN and END are special patterns that match the beginning and end of file (analogous to the meaning of ^ and $ in horizontal direction).

If you want something to execute before reading the file, use BEGIN. For instance, initialization of counters or something. END could then collect the results.

Answer 4

In the example given, which I think is simplified for pedagogic clarity, you are right that it is superfluous. You could get the same results without using BEGIN.

1 == NR            { print "Files found:\n" }
/\<[a|x].*\.conf$/ { print $9               }

would produce the same results since the print statement is constrained to just the first line of input.

That being said, the BEGIN and END blocks are incredibly powerful tools. As other solutions have mentioned, you can use the BEGIN block to initialise variables or other routines which only need to be performed once but it can also be used to run Awk commands when there are no files to process. A simple example:

BEGIN { print sqrt(12/4) }

You can see a more serious example of programming in Awk without processing any input here.

Likewise, the END block is extremely useful for performing calculations and summarising all of the input. This can't be done (usually) without first reading in all of the data. A simple example of summarising input can be found here