2

Is it possible to copy the rows of File1 and File2 in a new File4 following the instruction given by File3 by using a simple bash script (sed? awk?). 

 File1: /*two or more columns, tab delimited*/
    AC    456324
    DC    689712
    GH    123677
    KL    236587

File2: /*two or more columns, tab delimited*/
    DC    AABBC TTYJU
    AC    DDDEE YYUKI
    KL    GGHHG QQSSD

 File3: /*one column*/
    AC
    DC

File4 /*tab delimited*/
    AC    456324    DDDEE   YYUKI
    DC    689712    AABBC   TTYJU

I'm actually performing this using Python dictionaries, and I wondered if you knew a simple way out.

Note: this question, in its simplest form, has already an answer in: Merging file by row

|improve this question
3

Here's an extension of the simpler case you linked to, using an array in awk

$ awk 'BEGIN{OFS="\t"} NR==FNR {a[$1]=$1;next;} \
    $1 in a {k=$1;$1="";a[k]=a[k]$0} END{for (i in a) print a[i]}' File3 File1 File2
AC      456324  DDDEE   YYUKI
DC      689712  AABBC   TTYJU

Unlike join, it doesn't require pre-sorting of the files on the key field.

|improve this answer
3

You can accomplish it fairly with the join command. 

$ join -j 1 <(sort file3) <(sort file1) > tmp ; \
    join -j 1 <(sort tmp) <(sort file2) | sed 's/ /\t/g' > file4
  • I first use join on file3 and file1 since file3 has the keys.
  • Now, I write the output of the above command to tmp and now again do the join on tmp and file2.
  • Now, as per the above command, the output will be in space delimited format.
  • As your requirement is tab delimited format, I fed this to sed command which replaces all spaces by tab.
  • Finally, I write it to file4 as per your requirement.
|improve this answer
  • 1
    I think there's a race condition in structuring your command like that, given that file4 might not have been written out in time, so that the 2nd join can make use of it. In my testing this command sometimes works and sometimes doesn't and the subshells spawned for either side of the pipe is the only thing I can think of that would cause this. – slm♦ Jul 10 '14 at 19:55
  • An easy fix is to change the pipe to a semicolon. – slm♦ Jul 10 '14 at 19:56
  • 1
    @slm, thanks for fixing the code. I appreciate it. I wanted to provide a working solution to the OP as the OP needed tab delimited format as the output. :) – Ramesh Jul 10 '14 at 19:58
  • You don't need a temporary file (and why sort it for the 2nd join when 1st join output is already sorted ?). join -j1 <(join -j1 <(sort file3) <(sort file1)) <(sort file2) should do it. – don_crissti Jul 19 '15 at 23:06
1

If by "simple" you mean that you don't have to write a lot of code, there are a lot of tools like csvkit or csvfix out there that can help you with this - in particular, have a look at the "join" sub-commands they both provide.

But if you meant that it must not have any extra requirements, I don't think there is a "simple" way. At least it's definitely not going to be a one-liner.

Besides, this isn't a problem that I'd try to solve using just bash plus the standard Unix utilities, anyway. For things like these, you're doing fine with Python, in my opinion - especially if the problem becomes more complex.

|improve this answer
  • It's simple and it's definitely a one-liner, see my comment under Ramesh's post. – don_crissti Jul 19 '15 at 23:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.