Compare and print data from different text files to one in Shell

Question

How can I compare and print data from different text files to one in Shell.

I have captures NAS details of three different boxes using SSH, now I need to combine all the three text files to one file and MOUNT NAME should be in the first column and if the same MOUNT were present in he three boxes then it should print in same line and if the MOUNT is presented only in BOX_B and BOX_C then MOUNT name should present in first column and Column for Box_A should be kept blank

Lets take two examples df_BoxA.txt and df_BoxB.txt and df_BoxC.txt

Example:

$cat df_BoxA.txt  
/logs/boxA      2G     1.2G     7.7G    62%             NAS:/logs/boxA
/data/boxA      2G     1.8G     2.0G    91%             NAS:/data/boxA 
/apps/boxA      2G     1.4G     5.7G    72%             NAS:/apps/boxA 
/data/java      1G     67M      9.3G    7%              NAS:/data/java
/home/admin     10G    4.6G     54G     46%             NAS:/home/admin
/admin/arch     10G    8.3G     19G     83%             NAS:/admin/arch
/apps/dist      10G    8.3G     19G     83%             NAS:/apps/dist



$cat df_BoxB.txt  
/logs/boxA      2G     1.2G     7.7G    62%             NAS:/logs/boxB
/data/boxA      2G     1.8G     2.0G    91%             NAS:/data/boxB 
/apps/boxA      2G     1.4G     5.7G    72%             NAS:/apps/boxB 
/home/user      40G    29.3G    107G    74%             NAS:/home/user1 
/data/java      1G     67M      9.3G    7%              NAS:/data/java
/home/admin     10G    4.6G     54G     46%             NAS:/home/admin
/apps/dist      10G    8.3G     19G     83%             NAS:/apps/dist


$cat df_BoxC.txt  
/logs/boxA      2G     1.2G     7.7G    62%             NAS:/logs/boxC
/data/boxA      2G     1.8G     2.0G    91%             NAS:/data/boxC 
/apps/boxA      2G     1.4G     5.7G    72%             NAS:/apps/boxC 
/home/user1     40G    29.3G    107G    74%             NAS:/home/user1 
/home/admin     10G    4.6G     54G     46%             NAS:/home/admin
/admin/arch     10G    8.3G     19G     83%             NAS:/admin/arch
/apps/dist      10G    8.3G     19G     83%             NAS:/apps/dist

After combining all the three files the result should be like

$cat result.txt 
/logs/boxA   2G     1.2G     7.7G    62% NAS:/logs/boxA 2G  1.2G  7.7G  62% NAS:/logs/boxB  2G   1.2G  7.7G  62% NAS:/logs/boxC
/data/boxA   2G     1.8G     2.0G    91% NAS:/data/boxA 2G  1.8G  2.0G  91% NAS:/data/boxB  2G   1.8G  2.0G  91% NAS:/data/boxC
/apps/boxA   2G     1.4G     5.7G    72% NAS:/apps/boxA 2G  1.4G  5.7G  72% NAS:/apps/boxB  2G   1.4G  5.7G  72% NAS:/apps/boxC 
/data/java   1G     67M     9.3G    7%   NAS:/data/java 1G  67M   9.3G  7%  NAS:/data/java
/home/admin  10G    4.6G     54G     46% NAS:/home/admin10G 4.6G  54G   46% NAS:/home/admin 10G  4.6G  54G   46% NAS:/home/admin
/admin/arch  10G    8.3G     19G     83% NAS:/admin/arch                                    10G  8.3G  19G   83% NAS:/admin/arch
/apps/dist   10G    8.3G     19G     83% NAS:/apps/dist 10G 8.3G  19G   83% NAS:/apps/dist  10G  8.3G  19G   83% NAS:/apps/dist
/home/user                                              40G 29.3G 107G  74% NAS:/home/user1 
/home/user1                                                                                 40G  29.3G 107G  74% NAS:/home/user1

I have tried of using pr command which is combining of the files which not the required result.

Also tried of using sdiff but unable to get result.

How can I solve this?

Answer 1

What you want to do requires a little programming:

#!/usr/bin/perl
# Program to join files of TAB separated data based on first key
# --J. Ziobro--: 11/2014
use strict;
my $f;
my %allLines;
my $maxColumns = 0;
my $fileNum    = 0;
my %keys;
foreach $f (@ARGV) {
    die "Could not open $f" unless open( F, $f );
    while (<F>) {
        chop;
        my ( $key, @line ) = split /\t/;    # assume tab separate all cols
        $maxColumns = ( $maxColumns > @line ) ? $maxColumns : @line;

        # allLines is indexed by KEY <tab> FileNumber
        $keys{$key} = 1;
        $allLines{ $key . "\t" . $fileNum } = join( "\t", @line );
    }
    $fileNum++;
}
foreach ( keys %keys ) {
    print $_;
    for ( $f = 0 ; $f < $fileNum ; $f++ ) {
        if ( exists $allLines{ $_ . "\t" . $f } ) {
            print "\t", $allLines{ $_ . "\t" . $f };
        }
        else {
            print "     " x $maxColumns;
        }
    }
    print "\n";
}

Ciao,//Z\

Answer 2

I'm pretty sure you're looking for join. Unfortunately, I'm not very good with it. I know there's a way to make it fill the fields the way you want as well, but so far I can only get the unpaired lines to print at the head of the line. join only joins two files at a time, and so the unpaired lines don't show where you want - at least, I think they don't. Your question is a little unclear to me - my apologies.

Anyway, to use join you must first sort the input on the join field - which is by default the first and what I went with here:

for f in file[123]
do sort <<IN >"$f"
$(cat "$f")
IN
done

Next, as I said, join only joins two files at a time so I joined the first two and piped that output to another join for the third:

join -a1 -a2 file[12] | 
join -a1 -a2 - file3  | 
column -t | sort -hk2,2

I also piped it through column and sort again for formatting. The results follow:

/data/java   1G   67M    9.3G  7%   NAS:/data/java   1G   67M   9.3G  7%   NAS:/data/java
/apps/boxA   2G   1.4G   5.7G  72%  NAS:/apps/boxA   2G   1.4G  5.7G  72%  NAS:/apps/boxB   2G   1.4G  5.7G  72%  NAS:/apps/boxC
/data/boxA   2G   1.8G   2.0G  91%  NAS:/data/boxA   2G   1.8G  2.0G  91%  NAS:/data/boxB   2G   1.8G  2.0G  91%  NAS:/data/boxC
/logs/boxA   2G   1.2G   7.7G  62%  NAS:/logs/boxA   2G   1.2G  7.7G  62%  NAS:/logs/boxB   2G   1.2G  7.7G  62%  NAS:/logs/boxC
/admin/arch  10G  8.3G   19G   83%  NAS:/admin/arch  10G  8.3G  19G   83%  NAS:/admin/arch
/apps/dist   10G  8.3G   19G   83%  NAS:/apps/dist   10G  8.3G  19G   83%  NAS:/apps/dist   10G  8.3G  19G   83%  NAS:/apps/dist
/home/admin  10G  4.6G   54G   46%  NAS:/home/admin  10G  4.6G  54G   46%  NAS:/home/admin  10G  4.6G  54G   46%  NAS:/home/admin
/home/user1  40G  29.3G  107G  74%  NAS:/home/user1
/home/user   40G  29.3G  107G  74%  NAS:/home/user1