2

I have the cut command I want that grabs the first word in each line of a file. I then want to put each word from the cut command into a foreach. I then want to do a grep command inside the body of the foreach to grep for that word in another file.

Something like this:

@array = (cut /tmp/10218.after -f1); 
foreach $word (@lines) { 
   grep $word /tmp/10218.before;
} 

Obviously the @array assignment doesn't work. How do I get around this?

I'm sure there are many ways I just don't know what they are or which is best or good enough.

migrated from serverfault.com Apr 17 '14 at 10:22

This question came from our site for system and network administrators.

  • 2
    ... what language? This looks like a mashup of bash, perl, and php? – Zypher Apr 15 '14 at 20:17
  • bash. i'm more comfortable with perl than bash so my psuedocode may look more like perl than bash. – Ramy Apr 15 '14 at 20:21
  • 1
    bash starts to fall apart nastily when trying to do arrays at all. Just pretend you can't. – Sobrique Apr 15 '14 at 20:25
4

In bash

while read -r word
do
    grep -q "$word" file.before
    if [ $? -ne "0" ]
    then
        echo "$word not in file"
     fi
done < <(cut -f1 -d" " file.after)

The -q to grep tells it to be quiet, you can then interrogate $? to see if there was a match 0 or not 1.

  • beautiful. plus 1. – Ramy Apr 16 '14 at 13:50
3

You'll want to do something more like this:

for i in $(cat /tmp/10218.after)
do
    grep $(echo ${i} | cut -f1) /tmp/10218.before
done

If you want to get a bit more fancy and output something if the grep fails you cand do something like:

for i in $(cat /tmp/10218.after)
do
    COUNT=grep -c $(echo ${i} | cut -f1) /tmp/10218.before
    if [[ ${COUNT} -eq 0 ]]
    then
        echo "${i}: Not Found"
    else
        echo "${i}: Found"
    fi
done
  • this is correct. Can I trouble you to tell me how to negate the grep. i.e. print a message (or something) when the grep fails. IOW I want to know if any of the words are NOT in the .before file. – Ramy Apr 15 '14 at 20:25
  • redirect the grep to /dev/null, and then test $?. – Sobrique Apr 15 '14 at 20:28
  • You could have it echo -n "$i: " before the grep, and that would show you visually whether anything you're looping through from .after is missing in .before. – Basil Apr 15 '14 at 20:33
  • like this, @Sobrique: grep $(echo ${i} | cut -f1) /tmp/10218.before > /dev/null 2>&1 ? – Ramy Apr 15 '14 at 20:40
  • @Basil, I think that would get too busy and easy to miss in the output. – Ramy Apr 15 '14 at 20:41
1

Use perl.

#!/usr/bin/perl

use strict;
use warnings;

my %words_to_find;

open ( my $input, "<", "/tmp/10218.after" );
while ( my $line = <$input> )
{
  my ( $word ) = ( $line =~ m/\A(\S+)\s/ );
  $words_to_find{$word}++;
}
close ( $input );

open ( my $search, "<", "/tmp/10218.before" ); 
while ( my $line = <$search> )
{
  foreach my $word ( key %words_to_find )
  { 
    if ( $line =~ m/$word/ )
    {
      print $line;
      last;
    }
  }
}
close ( $search );

Something like this should do the trick.

  • a lot of "something like this"'s in this thread! – Michael Martinez Apr 15 '14 at 20:34
0

What your code seems to be doing is to extract the first field in a tab-delimited list in one file, and then attempting to find those words in a second file.

You can simplify this somewhat by not storing the list of words in an array:

cut -f1 /tmp/10218.after | grep -f /dev/stdin /tmp/10218.before

This would extract the words form the first file, and then pass them directly to grep as patterns to be used in matching against the second file.

There's a few optimisations we can do here though. First of all, we can make sure that the list of words only contain unique words:

cut -f1 /tmp/10218.after | sort -u | grep -f /dev/stdin /tmp/10218.before

Secondly, we can make sure that grep does string comparisons instead of regular expression matches:

cut -f1 /tmp/10218.after | sort -u | grep -F -f /dev/stdin /tmp/10218.before

And then, we possibly don't want grep to return matches for substrings (such as bee in bumblebee):

cut -f1 /tmp/10218.after | sort -u | grep -wF -f /dev/stdin /tmp/10218.before

We could also make sure that we only match the words in the first column of the second file by rewriting the words as anchored regular expressions (and drop -F):

cut -f1 /tmp/10218.after | sort -u | sed 's/^/^/' | grep -w -f /dev/stdin /tmp/10218.before

The sed command simply inserts ^ at the start of each line, so that instead of the string bee we get the regular expression ^bee.


Or, we could just employ a single awk program to do everything for us:

awk -F '\t' 'FNR == NR { words[$1]++; next } words[$1]' /tmp/10218.after /tmp/10218.before

This reads the first tab-delimited column of the first file into the array words as keys, and then checks the the words in the second file against these keys. If a word in the second file occurs as a key, the line form the second file is printed.


If you don't care about the ordering of the output, you could also use join:

join <( cut -f1 /tmp/10218.after | sort -u -b ) <( sort -b /tmp/10218.before )

This particular way of writing the command requires a shell (such as bash) that knows about process substitutions with <(...).

In other shells:

cut -f1 /tmp/10218.after | sort -u -b -o keys
sort -b -o data /tmp/10218.before
join keys data

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.