Count CSV columns and remove

Question

I have multiple csv files with the prefix mv. My goal is to remove specific column (named, or by number) from files containing 5 columns (but not other columns).

for f in mv*.csv
do
    ln= $(echo "$f" | awk '{print NF}' | sort -nu | head -n 1)
    if [ln==4]
    then
        $(echo "$f" cut -d, -f2,4 --complement >> $f)
    fi
done

The 3rd line throws a "command not found" error, so I haven't even gotten to test my "cut" part. What am I missing?

PS - I'll be happy to understand my mistake to know bash better, but any solution using sed/awk only/python/whatever would also be great.

Thanks

Answer 1

You pass the file name to awk, but you should pass its contents:

ln= $(cat "$f" | awk '{print NF}' | sort -nu | head -n 1)

or better:

ln= $(awk '{print NF}' "$f" | sort -nu | head -n 1)

Answer 2

Using csvcut from csvkit:

$ csvcut -C 5 data.csv >data-new.csv

This will remove column 5 from data.csv and save the result into data-new.csv.

$ csvcut -C "comment" data.csv >data-new.csv

This will remove the column named comment (according to the header line in the file) from data.csv and save the result into data-new.csv.