I have a command which looks like this:

cat PGC2.SCZ.1.dat | awk 'NR == 1 || $NF < 0.05/1783'

so this part I guess means skip the first line

awk 'NR == 1

but what does this refer to?

|| $NF < 0.05/1783'



NR means "number of record" and refers to the line streamed to awk. (By default every new, not empty line is a new record. One can define a different record separator RS. Then the term line isn't correct here anymore.)

NF means "number of fields" and refers to the number of columns in the line. Due to the $ before the NF, we are asking for the value in the last column. (NB: By default any run of spaces or tabs is used as a column delimiter aka field separator FS)

The || means "or".

So in summary your command will print out the first line in PGC2.SCZ.1.dat and all lines where the value in the last column is less then 0.05/1783.

|improve this answer|||||

First of all, since you're learning *nix, this is an example of a "useless use of cat" (a famous trope in the *nix world). The cat file | awk command isn't needed here, you could just do awk command file.

Anyway, on to the awk. The default action when an expression evaluates to true in awk is to print the current line. NR is the current line number, so NR == 1 will be true for the first line of the file. Then, awk splits its input into fields on whitespace (or anything else you choose using the -F option). The fields are then available as variables $1, $2, ... $NF, where NF is the number of fields. Therefore $NF is the contents of the last field.

Putting all that together, the expression awk 'NR == 1 || $NF < 0.05/1783' will be true on the first line and on any other line whose last field is less than 0.05 divided by 1783.

You could write the same thing less idiomatically, but more clearly, as:

awk '{ 
        if(NR ==1 ){
        else if ($NF < 0.05/1783){ 
     }' PGC2.SCZ.1.dat
|improve this answer|||||

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.