how to copy a column from another file when the ID's matches (multiple files in each folder)

Question

I just encountered with a question. It is very similar to what I already asked before how to copy a column from another file when the ID's matches, with some differences.

I have a folder (folder1), with multiple files:

desired_pos142535974
desired_pos142540918
desired_pos142541687
desired_pos142541814
desired_pos142541910
desired_pos142542976

head file looks like this

head desired_pos142535974
1   142535974   196 HG00100
1   142535974   64 HG00101
1   142535974   63 HG00103
1   142535974   26 HG00108
1   142535974   85 HG00110
1   142535974   83 HG00114
1   142535974   148 HG00115

I have another folder (folder2)

desired_output_pos142535974_g
desired_output_pos142540918_g
desired_output_pos142541687_g
desired_output_pos142541814_g
desired_output_pos142541910_g
desired_output_pos142542976_g

each file in the folder2 looks like this

head desired_output_pos142535974_g
    HG00096 0|1
    HG00097 1|0
    HG00099 0|1
    HG00100 1|0
    HG00101 0|1
    HG00102 0|1
    HG00103 1|1

each file in folder 1 has an identical file in the folder 2 with the same "pos...." ID (for example desired_pos142535974 and desired_output_pos142535974_g both belong to the same individual).

So, this time I want to check if the last column of a file in folder 1 matches the first column of a file in folder 2 with the identical "pos" ID, add the second column of file 2 to file1, ending up with something like

head desires_pos142535974
1   142535974   196 HG00100  1|0
1   142535974   64 HG00101   0|1
1   142535974   63 HG00103   1|1

Answer 1

I see the only problem you might faced is you want to feed same files name. Here you go:

$ find /path/to/dir1 -type f -name 'desired_pos*' -exec sh -c '
    awk "FNR==NR{seen[\$1]=\$2; next} seen[\$NF]{print \$0, seen[\$NF]}" \
        ./path/to/dir2/desired_output_${1#*_}_g $1 >/log/to/file/desires_${1#*_}' _ {} \;
1   142535974   196 HG00100 1|0
1   142535974   64 HG00101 0|1
1   142535974   63 HG00103 1|1

find only files -type f located in /path/to/dir1 path which their name starts with desired_pos* (* means followed by anything or none) then do the same as explained here; just here file2 is the file in /path/to/dir2/desired_output_xxx and $1 after that is file1 found in /path/to/dir1 where both files has same pos# parts. ${1#*_} is returning the same part which removes shortest match from file1 up-to first _ seen.