4

I have the cut command I want that grabs the first word in each line of a file. I then want to put each word from the cut command into a foreach. I then want to do a grep command inside the body of the foreach to grep for that word in another file.

Something like this:

@array = (cut /tmp/10218.after -f1); 
foreach $word (@lines) { 
   grep $word /tmp/10218.before;
} 

Obviously the @array assignment doesn't work. How do I get around this?

I'm sure there are many ways I just don't know what they are or which is best or good enough.

3
  • 2
    ... what language? This looks like a mashup of bash, perl, and php?
    – Zypher
    Commented Apr 15, 2014 at 20:17
  • bash. i'm more comfortable with perl than bash so my psuedocode may look more like perl than bash.
    – Ramy
    Commented Apr 15, 2014 at 20:21
  • 1
    bash starts to fall apart nastily when trying to do arrays at all. Just pretend you can't.
    – Sobrique
    Commented Apr 15, 2014 at 20:25

4 Answers 4

6

In bash

while read -r word
do
    grep -q "$word" file.before
    if [ $? -ne "0" ]
    then
        echo "$word not in file"
     fi
done < <(cut -f1 -d" " file.after)

The -q to grep tells it to be quiet, you can then interrogate $? to see if there was a match 0 or not 1.

1
  • beautiful. plus 1.
    – Ramy
    Commented Apr 16, 2014 at 13:50
4

You'll want to do something more like this:

for i in $(cat /tmp/10218.after)
do
    grep $(echo ${i} | cut -f1) /tmp/10218.before
done

If you want to get a bit more fancy and output something if the grep fails you cand do something like:

for i in $(cat /tmp/10218.after)
do
    COUNT=grep -c $(echo ${i} | cut -f1) /tmp/10218.before
    if [[ ${COUNT} -eq 0 ]]
    then
        echo "${i}: Not Found"
    else
        echo "${i}: Found"
    fi
done
5
  • this is correct. Can I trouble you to tell me how to negate the grep. i.e. print a message (or something) when the grep fails. IOW I want to know if any of the words are NOT in the .before file.
    – Ramy
    Commented Apr 15, 2014 at 20:25
  • redirect the grep to /dev/null, and then test $?.
    – Sobrique
    Commented Apr 15, 2014 at 20:28
  • You could have it echo -n "$i: " before the grep, and that would show you visually whether anything you're looping through from .after is missing in .before.
    – Basil
    Commented Apr 15, 2014 at 20:33
  • like this, @Sobrique: grep $(echo ${i} | cut -f1) /tmp/10218.before > /dev/null 2>&1 ?
    – Ramy
    Commented Apr 15, 2014 at 20:40
  • @Basil, I think that would get too busy and easy to miss in the output.
    – Ramy
    Commented Apr 15, 2014 at 20:41
1

Use perl.

#!/usr/bin/perl

use strict;
use warnings;

my %words_to_find;

open ( my $input, "<", "/tmp/10218.after" );
while ( my $line = <$input> )
{
  my ( $word ) = ( $line =~ m/\A(\S+)\s/ );
  $words_to_find{$word}++;
}
close ( $input );

open ( my $search, "<", "/tmp/10218.before" ); 
while ( my $line = <$search> )
{
  foreach my $word ( key %words_to_find )
  { 
    if ( $line =~ m/$word/ )
    {
      print $line;
      last;
    }
  }
}
close ( $search );

Something like this should do the trick.

1
  • a lot of "something like this"'s in this thread! Commented Apr 15, 2014 at 20:34
0

What your code seems to be doing is to extract the first field in a tab-delimited list in one file, and then attempting to find those words in a second file.

You can simplify this somewhat by not storing the list of words in an array:

cut -f1 /tmp/10218.after | grep -f /dev/stdin /tmp/10218.before

This would extract the words form the first file, and then pass them directly to grep as patterns to be used in matching against the second file.

There's a few optimisations we can do here though. First of all, we can make sure that the list of words only contain unique words:

cut -f1 /tmp/10218.after | sort -u | grep -f /dev/stdin /tmp/10218.before

Secondly, we can make sure that grep does string comparisons instead of regular expression matches:

cut -f1 /tmp/10218.after | sort -u | grep -F -f /dev/stdin /tmp/10218.before

And then, we possibly don't want grep to return matches for substrings (such as bee in bumblebee):

cut -f1 /tmp/10218.after | sort -u | grep -wF -f /dev/stdin /tmp/10218.before

We could also make sure that we only match the words in the first column of the second file by rewriting the words as anchored regular expressions (and drop -F):

cut -f1 /tmp/10218.after | sort -u | sed 's/^/^/' | grep -w -f /dev/stdin /tmp/10218.before

The sed command simply inserts ^ at the start of each line, so that instead of the string bee we get the regular expression ^bee.


Or, we could just employ a single awk program to do everything for us:

awk -F '\t' 'FNR == NR { words[$1]++; next } words[$1]' /tmp/10218.after /tmp/10218.before

This reads the first tab-delimited column of the first file into the array words as keys, and then checks the the words in the second file against these keys. If a word in the second file occurs as a key, the line form the second file is printed.


If you don't care about the ordering of the output, you could also use join:

join <( cut -f1 /tmp/10218.after | sort -u -b ) <( sort -b /tmp/10218.before )

This particular way of writing the command requires a shell (such as bash) that knows about process substitutions with <(...).

In other shells:

cut -f1 /tmp/10218.after | sort -u -b -o keys
sort -b -o data /tmp/10218.before
join keys data

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .