Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

Having a file of the following contents:

1111,2222,3333,4444
aaaa,bbbb,cccc,dddd

I seek to get a file equal to the original but lacking a n-th column like, for n = 2 (or may it be 3)

1111,2222,4444
aaaa,bbbb,dddd

or, for n = 0 (or may it be 1)

2222,3333,4444
bbbb,cccc,dddd

A real file can be gigabytes long having tens thousands columns.

As always in such cases, I suspect command line magicians can offer an elegant solution... :-)

In my actual real case I need to drop 2 first columns, which can be done by dropping a first column twice in a sequence, but I suppose it would be more interesting to generalise a bit.

share|improve this question
    
Are the fields guaranteed to not contain ,? (I.e., , is only ever used as a field separator.) –  Michael Kjörling Mar 21 '12 at 11:59
    
@MichaelKjörling, it would be nice to have a more flexible solution, but in my case - yes: the separator is , and it never occurs inside a field. –  Ivan Mar 21 '12 at 16:38
    
In that case, Scott's answer should be just the thing. –  Michael Kjörling Mar 21 '12 at 19:37

2 Answers 2

up vote 22 down vote accepted

I believe this is specific to cut from the GNU coreutils:

$ cut --complement -f 3 -d, inputfile
1111,2222,4444
aaaa,bbbb,dddd

Normally you specify the fields you want via -f, but by adding --complement you reverse the meaning, naturally. From 'man cut':

--complement
    complement the set of selected bytes, characters or fields

One caveat: if any of the columns contain a comma, it will throw cut off, because cut isn't a CSV parser in the same way that a spreadsheet is. Many parsers have different ideas about how to handle escaping commas in CSV. For the simple CSV case, on the command line, cut is still the way to go.

share|improve this answer
4  
That works fine as long as it's a simple CSV file. If any of the columns is a string with a comma in it, it'll throw cut off because it's not a CSV parser. If a CSV field has a field separator in its value, it's wrapped in quotes. Btw, on the subject of cut, -f takes field ranges. cut -f, -d3- will output the third field on, removing the first two. –  Alexios Mar 21 '12 at 11:37
2  
You mean cut -d, -f3- –  Useless Mar 21 '12 at 11:54
    
@Alexios that's a good point. I never really deal with "real" CSV, only the simple subset. I will edit my answer to reflect that. –  Scott McClung Mar 21 '12 at 16:46
    
@Useless: Damn, yes. That's what I call my ‘cut dyslexia’ striking again. sigh. Scott: CSV files are tricky beasts. Far too many different sub-formats, some of which aren't even CSV, but are conventionally called that anyway. –  Alexios Mar 21 '12 at 20:41
    
This prints the new CSV to my terminal - how do I get it to overwrite the input (or perhaps write to a new file, seems like OP was looking for either)? –  Max Ghenis Oct 22 at 16:33

If the data is simply made of comma-separated columns:

cut -d , -f 1-2,4-

You can also use awk, but it's a bit awkward because while clearing a field is easy, removing the separator takes some work. If you have no empty field, it's not too bad:

awk -F , 'BEGIN {OFS=FS}  {$3=""; sub(",,", ","); print}'

If you have actual CSV, where commas can appear inside fields if properly quoted, you need a real CSV library.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.