-1

This question already has an answer here:

Consider two csv files:

$ cat given.csv
123,John
246,Paul
369,George
987,Ringo

$ cat family.csv
246,McCartney
123,Lennon
987,Starr
369,Harrison

Note that the lines are not ordered! How might I use awk, grep, and other bash tools to get a combined output file (order unimportant):

123,John,Lennon
246,Paul,McCartney
369,George,Harrison
987,Ringo,Starr

I was thinking about running a for loop over the first file, then using awk to get the id and then grepping the second file for the relevant record. Is there a more straightforward way to do this in Bash? I find that often bash has a clever, efficient way to handle text files that I'm not yet familiar with.

marked as duplicate by Anthon, slm Sep 21 '14 at 12:50

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

  • The answers in the link you give use grep resp. awk. That has nothing to do with bash apart from bash being one of an unendless number of programs from which you could do so. – Anthon Sep 21 '14 at 10:27
  • Thank you Anthon. I don't mind the downvote if the question can be improved. Actually, I appreciate it so long as the downvoter mentions what needs improving! I mention bash as the for loop is done by Bash. – dotancohen Sep 21 '14 at 10:32
  • The simple joining of unordered files has been asked and answered before. The answer you selected doesn't work on CSV files in general, it just works for the restricted example you gave. The next person using more real CSV files going to find out the hard way. That makes the question not very useful. – Anthon Sep 21 '14 at 10:57
  • I see, Anthon. I don't see this as a dup of the link that you posted as the input formats are completely different (the OP of that question is not taking data from the second file's rows), even if the join command does resolve both issues. I actually did do a comprehensive search before asking and was surprised that this question had not been asked before. My original worry would be that the answer would be in perl! – dotancohen Sep 21 '14 at 11:58
  • Why worry about that? Perl has a library (as do Python, Ruby) that can properly handle CSV files, for which the join "solution" would give faulty results. – Anthon Sep 21 '14 at 12:09
4

With join and sort:

join -t , <(sort given.csv) <(sort family.csv)

Output:

123,John,Lennon
246,Paul,McCartney
369,George,Harrison
987,Ringo,Starr


With grep:

#!/bin/bash

F1="given.csv"
F2="family.csv"
D=","           # delimiter

while IFS="$D" read FIRST_COLUMN REST; do
  T="$FIRST_COLUMN$D$REST"
  T+="$(grep -oP "^$FIRST_COLUMN\K$D.*" "$F2")" && echo "$T"
done < "$F1"

Output:

123,John,Lennon
246,Paul,McCartney
369,George,Harrison
987,Ringo,Starr
  • Thank you. The join command is perfect! Note that the bash breaks if lines are found in the given.csv file that have no match in the last.csv file, though that requirement was not specified in the OP. No mind that, answer accepted! – dotancohen Sep 21 '14 at 10:21
  • Thanks for the note. Now I've fixed this bug in my bash/grep version. – Cyrus Sep 21 '14 at 11:04

Not the answer you're looking for? Browse other questions tagged or ask your own question.