Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Is it possible to copy the rows of File1 and File2 in a new File4 following the instruction given by File3 by using a simple bash script (sed? awk?).

 File1: /*two or more columns, tab delimited*/
    AC    456324
    DC    689712
    GH    123677
    KL    236587

File2: /*two or more columns, tab delimited*/

 File3: /*one column*/

File4 /*tab delimited*/
    AC    456324    DDDEE   YYUKI
    DC    689712    AABBC   TTYJU

I'm actually performing this using Python dictionaries, and I wondered if you knew a simple way out.

Note: this question, in its simplest form, has already an answer in: Merging file by row

share|improve this question
up vote 3 down vote accepted

Here's an extension of the simpler case you linked to, using an array in awk

$ awk 'BEGIN{OFS="\t"} NR==FNR {a[$1]=$1;next;} \
    $1 in a {k=$1;$1="";a[k]=a[k]$0} END{for (i in a) print a[i]}' File3 File1 File2
AC      456324  DDDEE   YYUKI
DC      689712  AABBC   TTYJU

Unlike join, it doesn't require pre-sorting of the files on the key field.

share|improve this answer

You can accomplish it fairly with the join command.

$ join -j 1 <(sort file3) <(sort file1) > tmp ; \
    join -j 1 <(sort tmp) <(sort file2) | sed 's/ /\t/g' > file4
  • I first use join on file3 and file1 since file3 has the keys.
  • Now, I write the output of the above command to tmp and now again do the join on tmp and file2.
  • Now, as per the above command, the output will be in space delimited format.
  • As your requirement is tab delimited format, I fed this to sed command which replaces all spaces by tab.
  • Finally, I write it to file4 as per your requirement.
share|improve this answer
I think there's a race condition in structuring your command like that, given that file4 might not have been written out in time, so that the 2nd join can make use of it. In my testing this command sometimes works and sometimes doesn't and the subshells spawned for either side of the pipe is the only thing I can think of that would cause this. – slm Jul 10 '14 at 19:55
An easy fix is to change the pipe to a semicolon. – slm Jul 10 '14 at 19:56
@slm, thanks for fixing the code. I appreciate it. I wanted to provide a working solution to the OP as the OP needed tab delimited format as the output. :) – Ramesh Jul 10 '14 at 19:58
You don't need a temporary file (and why sort it for the 2nd join when 1st join output is already sorted ?). join -j1 <(join -j1 <(sort file3) <(sort file1)) <(sort file2) should do it. – don_crissti Jul 19 '15 at 23:06

If by "simple" you mean that you don't have to write a lot of code, there are a lot of tools like csvkit or csvfix out there that can help you with this - in particular, have a look at the "join" sub-commands they both provide.

But if you meant that it must not have any extra requirements, I don't think there is a "simple" way. At least it's definitely not going to be a one-liner.

Besides, this isn't a problem that I'd try to solve using just bash plus the standard Unix utilities, anyway. For things like these, you're doing fine with Python, in my opinion - especially if the problem becomes more complex.

share|improve this answer
It's simple and it's definitely a one-liner, see my comment under Ramesh's post. – don_crissti Jul 19 '15 at 23:08

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.