1

I'm trying to write a simple file sanity check script. I have a directory with dozen CSV files containing id,edname,firstname,lastname,suffix,email.

I like to write a awk script to check if first field contain a number and is not empty. and fields number 3,4 & 6 are not empty and that the file contains 6 fields no more no less than 6, if all of this conditions are true nothing happens but if any of these conditions failed, re-name the file to .bad. here is wha i have but is not picking up missing values in columns 4,6.

for f in *.csv; do 
  awk -F, '!(NF==6 && $1+0==$1 && $3$4$6!=""){f=1; exit} END{exit f}' "$f" || mv "$f" "$f".bad;
done
2
  • 1
    Surely $3$4$6!="" will return true if at least one of the fields is non-empty. Is that what you really want? Commented Dec 7, 2017 at 18:36
  • no. I see. wha I want is, if any of these fields is empty then that's a bad file, because those fields are required/mandatory. Commented Dec 7, 2017 at 18:48

2 Answers 2

2

As steeldriver pointed out in the comments, your third test will be true if any of the three fields isn't empty. I assume you actually want something like this:

for f in *.csv; do 
  awk -F, '!(NF==6 && $1+0==$1 && $3!="" && $4!="" && $6!=""){f=1; exit} 
            END{exit f}' "$f" || mv "$f" "$f".bad;
done
1
  • how can I also check if $6 contains an @ sign? Commented Dec 8, 2017 at 2:52
1

No need to do string comparisons if you just want to ensure it's not null. Empty strings are falsy, so:

awk -F, 'BEGIN {flag=0} !(NF==6 && $1+0==$1 && $3 && $4 && $6) {flag=1} END {exit flag}'
4
  • This fails if fields 3, 4 or 6 are 0 though. But that's not empty.
    – terdon
    Commented Dec 7, 2017 at 19:13
  • I believe it fair to infer that a first name, last name, and email address of 0 are invalid, but you also raise a fair point.
    – DopeGhoti
    Commented Dec 7, 2017 at 19:23
  • lol, I had completely missed that point. You're right, a last name of 0 doesn't seem very likely :) Ah, but you should also check that $1 is a number.
    – terdon
    Commented Dec 7, 2017 at 19:24
  • yes $1 must be a number, otherwise is a bad file. Commented Dec 7, 2017 at 19:36

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .