I have a large CSV file (~15GB) which a client has sent over as an export of a table. The format is a follows:
field, ... , field, field which sometimes contains commas,
another field which sometimes contains commas, field, ... , field
Consequently, I can't import the data because of the shift produced in some rows where the fields contain one or more commas.
I do not need the data in those two fields, so getting rid of them would probably be the best option. Consequently, to avoid the two fields in the middle that cause trouble, I was thinking of something that for each row would take the first 9 fields (the ones before the two) and the last 57 fields (the ones after the two).
I'm guessing this is something that can be done with sed
or awk
but my expertise with these tools (or whatever may be better) is rather limited.