I searched for this task, and found the following older questions:
- Removing Duplicates from a CSV based on specified columns
- Identify unique records on CSV based on specific columns
But I can't use
awk because my data is a complex CSV file with multiple nested double quotes.
Let's say I want to deduplicate the following (simplified case):
Ref,xxx,zzz ref1,"foo, bar, base",qux ref1,"foo, bar, base",bar ref2,aaa,bbb
In the output I need it as follows:
Ref,xxx,zzz ref1,"foo, bar, base",qux ref2,aaa,bbb
awk solution, please, only with any CSV parser.
I tried the following:
mlr --csv uniq -a -g Ref file.csv
But it's an error.