How to count occurrences of each word belonging to a file in all of `n` number of files passed as arguments?

Question

I am looking for a shell script that accepts a list of file names as its arguments, counts and reports the occurrence of each word that is present in the first argument file on other argument files.

I am pretty much sure how to count the occurrences of a word with respect to one file.

That is by using this trick :

$ tr ' ' '\n' < FILE | grep -c WORD

I am stuck when it gets to n number of files.

This is what I have come with so far :

#!/bin/bash

if [ $# -lt 2 ]
    then
    echo "Very less arguments bro."
fi

 search_file=`tr '\n' ' ' < $1` # Make the first file in to a sequence of words.

for other_file in "$@"
do
    if [ $other_file = $1 ]
        then 
        continue
    fi

    # Modify this file such that each space turns in to a newline
    tr ' ' '\n' < $other_file > new_temp_file

    for search_word in $search_file
    do
        word_freq=`grep -c $search_word new_temp_file`
        echo "Word=$search_word Frequency=$word_freq"
    done
done

Answer 1

I'd do:

#! /bin/sh -
# usage: wordcount <file-with-words-to-search-for> [<file>...]
words=$(tr -s '[[:space:]]' '[\n*]' < "${1?No word list provided}" | grep .)
[ -n "$words" ] || exit

shift
for file do
  printf 'File: %s\n' "$file"
  tr -s '[[:space:]]' '[\n*]' | grep -Fxe "$words" | sort | uniq -c | sort -rn
done

(that only gives a count for the words that are found at least once in each file).

Answer 2

You can iterate across a list of files provided on the command line like this:

for file in "$@"
do
    echo "Considering file ==> $file <=="
done

Your method for matching words should be perfectly effective. You can also search for occurrences of a word by using grep -o

echo 'I can cry cryogenic tears when I scry my hands. Can you cry too?' |
    grep -o '\bcry\b'    # \b marks a word boundary

Piping the result of that into wc -l will give you the number of occurrences in the input stream.

Using $( ... ) allows one to interpolate the output of a command into the text used by another. For example

echo "The date and time right now is $(date)"

We need some extra work to avoid searching the first file, but instead using that as the list of words. But putting this together you can end up something like this:

wordfile="$1"
wordlist=($(cat "$wordfile"))
shift

for file in "$@"
do
    for word in "${wordlist[@]}"
    do
        # echo "$file: $word:" $(grep -o "\b${word}\b" "$file" | wc -l)  # My way
        echo "$file: $word:" $(tr ' ' '\n' <"$file" | grep -c "$word")   # Your way
    done
done

It's not terribly efficient because for N words it will search each file N times. You might find that grep -f is of assistance here.

Answer 3

fgrep -cw 'word' file1 file2 ... fileN

That will output the following:

file1:4
file2:16

and so on, one per line. If it's just the total of all files, then do something like this:

echo "Total: $(( $(fgrep -cw 'word' file1 file2 ... fileN | awk -F: '{ print $NF" + " }') 0 ))"

which would output:

Total: 20