1

I need to interrogate the header of a CSV file, and if a column exists proceed with the data rows. Context is when the data contains columns depending on when and what emitted it.

Hoping for a "pure" Awk solution to keep business logic in a common language, but if this is not possible, interested in approaches that selectively deliver files with the header match to the Awk script.

Using latest version of Gawk is always an option.

Edit to add pseudo code:

if column in header (NR==1): then proceed with rest of file, else stop processing file

  • awk certainly knows (FNR) the current line number; do you know or care which column needs to match, or just that it exists? – Jeff Schaller Feb 15 '16 at 17:26
  • In this case, if the column is in the header, I can proceed irrespective of its position. Added process description above. – kermatt Feb 15 '16 at 17:51
2

to list files:

try

awk 'FNR == 1 && $4 == "whatever" { print FILENAME ;}' file1 ... filen |

which will select all file having whatever in fouth colum.

If you have funny name, just add quotes.

awk 'FNR == 1 && $4 == "whatever" { printf "\"s\"\n", FILENAME ;}' file1 ... filen |

to process one file

awk 'NR == 1 && $4 != "whatever" { exit ;}  other patterns { other action;}' file

to process many file

awk 'NR == 1 && $4 != "whatever" { nextfile ;}  other patterns { other action;}' file1 ... filen

which could be read as

  • IF (condition not met) NR == 1 && $4 != "whatever"
  • THEN skip this this file { nextfile ;}
  • ELSE proceed other patterns { other action;}
  • That lists the files to process, I am looking to process the files when the header match occurs. I could use one script to supply another with a list of files to process, but hoping for a "cleaner" one script solution. – kermatt Feb 15 '16 at 18:52
2

Assuming a simple, comma-delimited file where every comma is a delimiter (some csv files may have quoted commas that ought not to be treated as field seperators), the following prints every line except the header when a column in the header is "SOMESTRING":

awk -F, '
    FNR==1 {
        for (i=1; i<=NF; i++) 
            if ($i == "SOMESTRING")
                next
        nextfile
    }
    1
' file1 file2 file3 file4

The string comparison can be replaced with a substring test or a regular expression match operation, if appropriate.

nextfile is not part of POSIX AWK, but it is widespread; it is available in at least gawk, nawk (used on *BSD systems), mawk, and busybox.

1

awk 'FNR==1 && ! /whatever/ { nextfile } ; ...remainder of awk script here...' list_of_files_to_process

This should skip to the next file to be processed unless "whatever" is on line 1.

I can't remember whether nextfile is a GNU awk extension or if it is available in other awks too. The mawk man page doesn't mention it, but it is mentioned in the man page for original-awk. If it matters to you, check it before relying on the feature.

If you're using GNU awk, you may want to put that test in a BEGINFILE block like:

 BEGINFILE { FNR==1 && ! /whatever/ { nextfile } } ;
 ... remainder of awk script here ...

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.