I have two folders each containing 500 files, folder 1:


and another folder (folder 2) with these


The first file in folder one corresponds to the first file in folder 2 and so on. I want to keep those lines from file 2 that the first column matches the first column of file one. If it was a single file, I would easily use

awk 'FNR==NR {a[$1]; next}; $1 in a' file1 file2 > file_match 

But I have 500 files, so I need to run it through looping. I do not know how to loop through 2 files! any suggestion? Thanks

  • 1
    You talk about columns, but you never say what column delimiter your using (and using the ordinary whitespace as delimiter, you only have one column in each file). Also, it would be good to know what the expected result should be. – Kusalananda Feb 9 '18 at 22:16
  • @Kusalananda I think what the user has posted are file names and they want to apply their awk command to pairs of such files, with one file taken from each directory...? – steeldriver Feb 9 '18 at 22:39
  • @steeldriver Huh, that's what I get for using my phone to read... Well that just means there's even more info missing. – Kusalananda Feb 9 '18 at 22:47
  • Maybe you have a look at the join command, in case awk is broken. :) – user unknown Feb 10 '18 at 0:15
  • Is the order of the various contents of *column 1*s relevant? Are the respective *column 1*s sorted? – agc Feb 10 '18 at 17:26

Store the files for each folder in arrays:

f1=( /path/to/folder1/* )
f2=( /path/to/folder2/* )

Then iterate over the numeric indices of the arrays

for idx in "${!f1[@]}"; do
    awk ... "${f1[$idx]}" "${f2[$idx]}" > "${f1[$idx]}.matched"
  • Since the names are nearly identical, maybe a check whether they match is appropriate. – user unknown Feb 10 '18 at 0:14
  • @glenn jackman, thanks for the tip, I eventually used for idx in "${!f1[@]}"; do awk 'FNR==NR {a[$1]; next}; $1 in a' "${f1[$idx]}" "${f2[$idx]}" > "${f1[$idx]}.matched_try1" done. and it gave my my desired output – Anna1364 Feb 12 '18 at 17:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.