4

I have two csv files, there are some overlap columns between these two files. Assume one file is called as A.csv and another is called as B.csv. The intersection of A and B is called as C.

I would like to generate three new csv files: C; the subset of B after subtracting C from it, which is called as D; the union of A and D, which is called as E.

Are there any ways to do that in Linux/Unix using command without applying heavy weight programming languages?

5
  • related question
    – jw013
    Commented Sep 7, 2012 at 20:03
  • Do some of the fields have quotes around them or embedded commas? If so, you need a “heavyweight” programming language. (Well, ok, you can get away with awk, but it's not pretty.) Commented Sep 7, 2012 at 22:11
  • No, they are just normal csv files. Every entry is either a word or a numerical value.
    – user785099
    Commented Sep 8, 2012 at 22:02
  • Could you add small sample input and resultant files??
    – Paddy3118
    Commented Sep 11, 2012 at 12:48
  • I would have to see the file in question for a solution but do not underestimate the power of UNIX join and sort.
    – terdon
    Commented Nov 30, 2012 at 12:52

1 Answer 1

2

I would use python for this, don't be intimidated by python it's great at this kind of thing. My (rough and untested) solution for your problem would be:

f_csv_1 = open("csv1.csv")
f_csv_2 = open("csv2.csv")

csv_1 = f_csv_1.readlines()
csv_2 = f_csv_2.readlines()

f_csv_1.close()
f_csv_2.close()

intersection = list(set(csv_1) & set(csv_2))
union = list(set(csv_1) | set(csv_2))

out_1 = open("intersection.csv", "w")

for row in intersection:
      out_1.write("%s" % row)

out_2 = open("union.csv", "w")

for row in union:
      out_2.write("%s" % row)

out_1.close()
out_2.close()
2
  • Remember about closing files. Commented Sep 9, 2012 at 7:10
  • @gorkypl fixed­
    – jsj
    Commented Sep 9, 2012 at 10:08

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .