I'm trying to write a simple file sanity check script. I have a directory with dozen CSV files containing id,edname,firstname,lastname,suffix,email.

I like to write a awk script to check if first field contain a number and is not empty. and fields number 3,4 & 6 are not empty and that the file contains 6 fields no more no less than 6, if all of this conditions are true nothing happens but if any of these conditions failed, re-name the file to .bad. here is wha i have but is not picking up missing values in columns 4,6.

for f in *.csv; do 
  awk -F, '!(NF==6 && $1+0==$1 && $3$4$6!=""){f=1; exit} END{exit f}' "$f" || mv "$f" "$f".bad;
  • 1
    Surely $3$4$6!="" will return true if at least one of the fields is non-empty. Is that what you really want? – steeldriver Dec 7 '17 at 18:36
  • no. I see. wha I want is, if any of these fields is empty then that's a bad file, because those fields are required/mandatory. – daniel caceres Dec 7 '17 at 18:48

As steeldriver pointed out in the comments, your third test will be true if any of the three fields isn't empty. I assume you actually want something like this:

for f in *.csv; do 
  awk -F, '!(NF==6 && $1+0==$1 && $3!="" && $4!="" && $6!=""){f=1; exit} 
            END{exit f}' "$f" || mv "$f" "$f".bad;
  • how can I also check if $6 contains an @ sign? – daniel caceres Dec 8 '17 at 2:52

No need to do string comparisons if you just want to ensure it's not null. Empty strings are falsy, so:

awk -F, 'BEGIN {flag=0} !(NF==6 && $1+0==$1 && $3 && $4 && $6) {flag=1} END {exit flag}'
  • This fails if fields 3, 4 or 6 are 0 though. But that's not empty. – terdon Dec 7 '17 at 19:13
  • I believe it fair to infer that a first name, last name, and email address of 0 are invalid, but you also raise a fair point. – DopeGhoti Dec 7 '17 at 19:23
  • lol, I had completely missed that point. You're right, a last name of 0 doesn't seem very likely :) Ah, but you should also check that $1 is a number. – terdon Dec 7 '17 at 19:24
  • yes $1 must be a number, otherwise is a bad file. – daniel caceres Dec 7 '17 at 19:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.