Using ksh88 on AIX machine, I've tried a whole slew of things that are ultimately not working.

File_A has 2 columns without headers: the user group that created the directory in column 1, and the full file path in column 2, e.g.,:

Userwh0c4r35     /fake/file/path/directory_name
User1234567      /another/file/path/different_dir
User0987654      /some/other/path/another_name

File_B has 2 columns without headers: the size of the directory in MB in column 1, and the partial path name of the directory in column 2, e.g.,:

2183.31     directory_name
1750.09     directory_name/subfolder
1028.14     directory_name/subfolder/sub_subfolder
3658.97     different_dir
2159.62     different_dir/subfolder
1001.01     different_dir/different_subfolder 


The catch is that there are duplicate directory names in File_B (i.e., directory_name, directory_name/subfolder, directory_name/subfolder/sub_subfolder, ...)

What I want is, in one file, this output (I honestly don't care about the order of the columns, just that they're all present):

Userwh0c4r35     /fake/file/path/directory_name     2183.31
User1234567      /another/file/path/different_dir   3658.97

It seems simple enough but I haven't been able to figure it out. The closest I can get is to get the user group and the full path name and the line numbers from both files where the partial match was found, but I can not get the directory size (column 1 from File_B) ...

The code that I've settled on that gets me so close but not quite there (cobbled together from SO and various online tutorials), is:

awk '
    for(i in a) 
        if($2 ~ i) 
            {print $2,a[$2],$1} 
}' file_B file_A 

which produces a list with duplicates for every line from File_A where column 2 from File_A has a partial match in column 2 from File_B, e.g.:

Userwh0c4r35     /fake/file/path/directory_name
Userwh0c4r35     /fake/file/path/directory_name
Userwh0c4r35     /fake/file/path/directory_name

(One each for directory_name, directory_name/subfolder, and directory_name/subfolder/sub_subfolder)

I've tried to print everything I could think of, but to no avail... NR,FNR,i,$0,a[$NR],a[$FNR],a[$1],a[$2],$1,$2 ... I've also tried to use printf but that didn't work either...

  • based on partial string match in one column - nice, you deserve an upvote just for that! Seeing an accurate statement of what you want to match on is a breath of fresh air after all the questions we see about matching on "patterns"! – Ed Morton Feb 20 at 13:55
  • You say The catch is that there are duplicate directory names in File_B but your 1 line of sample input and expected doesn't encompass that catch. Please edit your question to show concise, testable sample input and expected output that covers that "catch" scenario and any others so we have something we can test a potential solution against and get a simple pass/fail result. – Ed Morton Feb 20 at 13:59
  • Will do Ed; apologies for any confusion - I see now how ambiguous my statement was. – nraslan Feb 22 at 0:15

What you want is essentially a join of two database tables. Conveniently, there is a command for that, aptly named join.

No need for awk here. Note that I have neither ksh88 nor AIX; this is Bash on Linux, but I quickly checked the AIX manual and think it should work.

I prepared this test environment:

$ cat filea
Userwh0c4r35     /fake/file/path/directory_name5
Userwh0c4r36     /fake/file/path/directory_name6
Userwh0c4r37     /fake/file/path/directory_name7
$ cat fileb
1234    directory_name5
2345    directory_name6
3456    directory_name7

Step 1: Add a column to filea consisting of only the last part of the directory pathname:

$ sed 's|\(.*\)/\(.*\)|\1/\2 \2|' filea > filea.tmp
$ cat filea.tmp
Userwh0c4r35     /fake/file/path/directory_name5 directory_name5
Userwh0c4r36     /fake/file/path/directory_name6  directory_name6
Userwh0c4r37     /fake/file/path/directory_name7  directory_name7

Step 2: Sort both files by the directory name (thanks, Mark Plotnick, for pointing this out):

$ sort -k3 filea.tmp > filea.tojoin
$ sort -k2 fileb > fileb.tojoin

Step 3: Use join to join this with fileb, based on the directory name in column 3 (filea) and column 2 (fileb):

$ join -1 3 -2 2 filea.tojoin fileb.tojoin > result
$ cat result
directory_name5 Userwh0c4r35 /fake/file/path/directory_name5 1234
directory_name6 Userwh0c4r36 /fake/file/path/directory_name6  2345
directory_name7 Userwh0c4r37 /fake/file/path/directory_name7  3456

Optional step 4: A cut will get rid of the first column, if you don't want it.

  • Not just likely, but surely. I had forgotten this little detail. – berndbausch Feb 20 at 11:28
  • This seems to have done it; double-checking now. Thank you so much!! – nraslan Feb 22 at 19:32

It's not clear what you're trying to do and your sample input/output isn't currently useful for testing with but here's one guess at it:

$ cat tst.awk
BEGIN { OFS="\t" }
    val = $1                                    # val = Userwh0c4r35 or 2183.31
    sub(/^[^[:space:]]+[[:space:]]+/,"")        # Allows spaces in directory names vs using $2
    dir = $0                                    # dir = /fake/file/path/directory_name or directory_name/subfolder
    sub(".*/","",dir)                           # dir = directory_name
    dir2path[dir] = $0
    dir2grp[dir]  = val
    sub("/.*","",dir)                           # dir = directory_name
    print dir2grp[dir], dir2path[dir], val

$ awk -f tst.awk File_A File_B
Userwh0c4r35    /fake/file/path/directory_name  2183.31
Userwh0c4r35    /fake/file/path/directory_name  12345
Userwh0c4r35    /fake/file/path/directory_name  9876

The above assumes that the same directory_name can't appear at the end of different paths in File_A (e.g. /foo/directory_name and /bar/directory_name) and was run on these input files:

$ head File_*
==> File_A <==
Userwh0c4r35     /fake/file/path/directory_name

==> File_B <==
2183.31     directory_name
12345       directory_name/subfolder
9876        directory_name/subfolder/sub_subfolder

If your directory names can contain tabs then you'd need to use a different output format. If they can contain newlines then you'll need a different input format too.

  • Ed, I really appreciate your taking the time to help; I regret that because I gave insufficient context in my question, your answer didn't solve the problem. – nraslan Feb 22 at 14:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.