I have a bash script to combine many files into a single csv file. This csv file has 47 fields and several 10000 rows.
To remove duplicates I was using
awk '!seen[$0]++' however, for sorting I have added a reference to the originating source in column 47.
I still only want to index on column 1 to 46 and print all including 47.
Must I list all 46 as in
awk '!seen[$1, $2, $3, ,..etc.., $45, $46]++' or is there an easier way?