how to pipe the output of cut to the foreach command?

Question

I have the cut command I want that grabs the first word in each line of a file. I then want to put each word from the cut command into a foreach. I then want to do a grep command inside the body of the foreach to grep for that word in another file.

Something like this:

@array = (cut /tmp/10218.after -f1); 
foreach $word (@lines) { 
   grep $word /tmp/10218.before;
} 

Obviously the @array assignment doesn't work. How do I get around this?

I'm sure there are many ways I just don't know what they are or which is best or good enough.

Answer 1

In bash

while read -r word
do
    grep -q "$word" file.before
    if [ $? -ne "0" ]
    then
        echo "$word not in file"
     fi
done < <(cut -f1 -d" " file.after)

The -q to grep tells it to be quiet, you can then interrogate $? to see if there was a match 0 or not 1.

Answer 2

You'll want to do something more like this:

for i in $(cat /tmp/10218.after)
do
    grep $(echo ${i} | cut -f1) /tmp/10218.before
done

If you want to get a bit more fancy and output something if the grep fails you cand do something like:

for i in $(cat /tmp/10218.after)
do
    COUNT=grep -c $(echo ${i} | cut -f1) /tmp/10218.before
    if [[ ${COUNT} -eq 0 ]]
    then
        echo "${i}: Not Found"
    else
        echo "${i}: Found"
    fi
done

Answer 3

Use perl.

#!/usr/bin/perl

use strict;
use warnings;

my %words_to_find;

open ( my $input, "<", "/tmp/10218.after" );
while ( my $line = <$input> )
{
  my ( $word ) = ( $line =~ m/\A(\S+)\s/ );
  $words_to_find{$word}++;
}
close ( $input );

open ( my $search, "<", "/tmp/10218.before" ); 
while ( my $line = <$search> )
{
  foreach my $word ( key %words_to_find )
  { 
    if ( $line =~ m/$word/ )
    {
      print $line;
      last;
    }
  }
}
close ( $search );

Something like this should do the trick.

Answer 4

What your code seems to be doing is to extract the first field in a tab-delimited list in one file, and then attempting to find those words in a second file.

You can simplify this somewhat by not storing the list of words in an array:

cut -f1 /tmp/10218.after | grep -f /dev/stdin /tmp/10218.before

This would extract the words form the first file, and then pass them directly to grep as patterns to be used in matching against the second file.

There's a few optimisations we can do here though. First of all, we can make sure that the list of words only contain unique words:

cut -f1 /tmp/10218.after | sort -u | grep -f /dev/stdin /tmp/10218.before

Secondly, we can make sure that grep does string comparisons instead of regular expression matches:

cut -f1 /tmp/10218.after | sort -u | grep -F -f /dev/stdin /tmp/10218.before

And then, we possibly don't want grep to return matches for substrings (such as bee in bumblebee):

cut -f1 /tmp/10218.after | sort -u | grep -wF -f /dev/stdin /tmp/10218.before

We could also make sure that we only match the words in the first column of the second file by rewriting the words as anchored regular expressions (and drop -F):

cut -f1 /tmp/10218.after | sort -u | sed 's/^/^/' | grep -w -f /dev/stdin /tmp/10218.before

The sed command simply inserts ^ at the start of each line, so that instead of the string bee we get the regular expression ^bee.

---

Or, we could just employ a single awk program to do everything for us:

awk -F '\t' 'FNR == NR { words[$1]++; next } words[$1]' /tmp/10218.after /tmp/10218.before

This reads the first tab-delimited column of the first file into the array words as keys, and then checks the the words in the second file against these keys. If a word in the second file occurs as a key, the line form the second file is printed.

---

If you don't care about the ordering of the output, you could also use join:

join <( cut -f1 /tmp/10218.after | sort -u -b ) <( sort -b /tmp/10218.before )

This particular way of writing the command requires a shell (such as bash) that knows about process substitutions with <(...).

In other shells:

cut -f1 /tmp/10218.after | sort -u -b -o keys
sort -b -o data /tmp/10218.before
join keys data