I have multiple csv files with the prefix mv. My goal is to remove specific column (named, or by number) from files containing 5 columns (but not other columns).

for f in mv*.csv
    ln= $(echo "$f" | awk '{print NF}' | sort -nu | head -n 1)
    if [ln==4]
        $(echo "$f" cut -d, -f2,4 --complement >> $f)

The 3rd line throws a "command not found" error, so I haven't even gotten to test my "cut" part. What am I missing?

PS - I'll be happy to understand my mistake to know bash better, but any solution using sed/awk only/python/whatever would also be great.


  • suggestion: [ $(head -n 1 "$f" | wc -w) -eq 4 ] || continue – Jeff Schaller Feb 3 '17 at 18:19

You pass the file name to awk, but you should pass its contents:

ln= $(cat "$f" | awk '{print NF}' | sort -nu | head -n 1)

or better:

ln= $(awk '{print NF}' "$f" | sort -nu | head -n 1)

Using csvcut from csvkit:

$ csvcut -C 5 data.csv >data-new.csv

This will remove column 5 from data.csv and save the result into data-new.csv.

$ csvcut -C "comment" data.csv >data-new.csv

This will remove the column named comment (according to the header line in the file) from data.csv and save the result into data-new.csv.

  • Great. I wasn't aware of csvkit. – Ozeuss Feb 3 '17 at 13:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.