Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have two csv files, there are some overlap columns between these two files. Assume one file is called as A.csv and another is called as B.csv. The intersection of A and B is called as C.

I would like to generate three new csv files: C; the subset of B after subtracting C from it, which is called as D; the union of A and D, which is called as E.

Are there any ways to do that in Linux/Unix using command without applying heavy weight programming languages?

share|improve this question
related question – jw013 Sep 7 '12 at 20:03
Do some of the fields have quotes around them or embedded commas? If so, you need a “heavyweight” programming language. (Well, ok, you can get away with awk, but it's not pretty.) – Gilles Sep 7 '12 at 22:11
No, they are just normal csv files. Every entry is either a word or a numerical value. – user785099 Sep 8 '12 at 22:02
Could you add small sample input and resultant files?? – Paddy3118 Sep 11 '12 at 12:48
I would have to see the file in question for a solution but do not underestimate the power of UNIX join and sort. – terdon Nov 30 '12 at 12:52

I would use python for this, don't be intimidated by python it's great at this kind of thing. My (rough and untested) solution for your problem would be:

f_csv_1 = open("csv1.csv")
f_csv_2 = open("csv2.csv")

csv_1 = f_csv_1.readlines()
csv_2 = f_csv_2.readlines()


intersection = list(set(csv_1) & set(csv_2))
union = list(set(csv_1) | set(csv_2))

out_1 = open("intersection.csv", "w")

for row in intersection:
      out_1.write("%s" % row)

out_2 = open("union.csv", "w")

for row in union:
      out_2.write("%s" % row)

share|improve this answer
Remember about closing files. – Paweł Rumian Sep 9 '12 at 7:10
@gorkypl fixed­ – jsj Sep 9 '12 at 10:08

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.