I have two files:


I'd like to create a command that identifies terms in terms_to_search.out that are not used in any of the files in files_to_search.out

Is there an easy way to do this?

2 Answers 2


quick ugly attempt at a one liner (with GNU grep for the -o option):

grep -of terms_to_search_out $(cat files_to_search.out | tr '\n' ' ') | sort | uniq | grep  -vf terms_to_search_out 
  • 1
    Note that all of \n, \t and space are used by default to split command substitutions, so there's no point changing '\n' to ' '. Restricting IFS to newline and do the reverse conversion would even make more sense since newline characters are less likely to occur in file names than space characters. Also note that filename generation is also performed (except in zsh) upon command substitution so blank characters in filenames are not the only problematic ones (see set -f to disable filename generation). Commented Nov 7, 2012 at 22:27
  • learn something new everyday - wasn't aware about the \n defaults. So maybe pass through sed to add quotes? or go with a for loop? Commented Nov 8, 2012 at 15:00

This is a bit tricky if you want to account for terms that can overlap, e.g. a single line containing banana is enough to count as a use of both ban and nan.

Here's a minimally-tested, quick-and-dirty perl script. It reads the strings to search (the needles) and the file names, then builds a regular expression that matches any of the needles. When it finds a match, it removes the matched string from the set of needles and rebuilds the regex. The needles that are left over at the end are the ones you're after.

#! /usr/bin/env perl
open FILENAMES, "<", "files_to_search.out" or die $!;
@filenames = <FILENAMES>;
chomp foreach @filenames;
open NEEDLES, "<", "terms_to_search.out" or die $!;
@needles = <NEEDLES>;
close NEEDLES;
chomp foreach @needles;
%needles = map {$_, 1} @needles;
sub build_re {
    $re = qr/(@{[join("|", map quotemeta, keys %needles)]})/;
@ARGV = @filenames;
while (<ARGV>) {
    while (/$re/) {
        delete $needles{$1};
        exit if !%needles;
print map "$_\n", sort keys %needles;

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .