Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have two files:


I'd like to create a command that identifies terms in terms_to_search.out that are not used in any of the files in files_to_search.out

Is there an easy way to do this?

share|improve this question
up vote 1 down vote accepted

This is a bit tricky if you want to account for terms that can overlap, e.g. a single line containing banana is enough to count as a use of both ban and nan.

Here's a minimally-tested, quick-and-dirty perl script. It reads the strings to search (the needles) and the file names, then builds a regular expression that matches any of the needles. When it finds a match, it removes the matched string from the set of needles and rebuilds the regex. The needles that are left over at the end are the ones you're after.

#! /usr/bin/env perl
open FILENAMES, "<", "files_to_search.out" or die $!;
@filenames = <FILENAMES>;
chomp foreach @filenames;
open NEEDLES, "<", "terms_to_search.out" or die $!;
@needles = <NEEDLES>;
close NEEDLES;
chomp foreach @needles;
%needles = map {$_, 1} @needles;
sub build_re {
    $re = qr/(@{[join("|", map quotemeta, keys %needles)]})/;
@ARGV = @filenames;
while (<ARGV>) {
    while (/$re/) {
        delete $needles{$1};
        exit if !%needles;
print map "$_\n", sort keys %needles;
share|improve this answer

quick ugly attempt at a one liner (with GNU grep for the -o option):

grep -of terms_to_search_out $(cat files_to_search.out | tr '\n' ' ') | sort | uniq | grep  -vf terms_to_search_out 
share|improve this answer
Note that all of \n, \t and space are used by default to split command substitutions, so there's no point changing '\n' to ' '. Restricting IFS to newline and do the reverse conversion would even make more sense since newline characters are less likely to occur in file names than space characters. Also note that filename generation is also performed (except in zsh) upon command substitution so blank characters in filenames are not the only problematic ones (see set -f to disable filename generation). – Stéphane Chazelas Nov 7 '12 at 22:27
learn something new everyday - wasn't aware about the \n defaults. So maybe pass through sed to add quotes? or go with a for loop? – Drake Clarris Nov 8 '12 at 15:00

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.