Look for English word in a file via terminal

Question

How to find and print out English words contained in a file via linux command line?

Thanks

Answer 1

GNU grep has the following options:

grep --only-matching --ignore-case --fixed-strings --file /usr/share/dict/british-english-insane /path/to/file.txt

This outputs strings found one-per-line. Here /usr/share/dict/british-english-insane is a wordlist provided by the Debian package wbritish-insane.

Answer 2

He, funny !

file=/usr/share/licenses/common/GPL3/license.txt
dict=/usr/share/dict/cracklib-small

while read word; do
    grep >/dev/null -i "\<$word\>" $file &&
        printf 'Word "%s" found in GPLv3...\n' $word
done < $dict

Output :

Word a found in GPLv3...
Word ability found in GPLv3...
Word about found in GPLv3...
(...)

The cracklib-small file comes with the package cracklib http://sourceforge.net/projects/cracklib

Answer 3

grep based solutions will generally be quite slow especially with large word lists.

You can take advantage from the fact that word lists are already sorted (however on my system, it seems at least the british-english one has been sorted in the POSIX/C locale even though it's UTF-8 encoded):

tr -cs "[:alpha:]'" '[\n*]' < /etc/passwd |
  LC_ALL=C sort -u |
  LC_ALL=C comm -12 - /usr/share/dict/british-english-insane

You may also want to convert everything lowercase or uppercase beforehand if you want to look for words in a case-insensistive manner.

Answer 4

file=/usr/lib/python2.6/LICENSE.txt
dict=/usr/share/dict/british-english-huge   # or any suitable list

sort "$dict" \
     <(sed "s/[].,\"?!;:#$%&()*+<>=@\^_{}|~[]\+/\n/g   # keep ' for now
            s|[-/[[:digit:][:blank:][:cntrl:]]\+|\n|g
            s/\<'\+/\n/; s/'\>\+/\n/                   # remove '
           " <(<"$file" tr '[:upper:]' '[:lower:]') ) |
uniq -c | awk '$1 > +1 {print $2}' 

found 382 words (case insensitive) in time:

real   0m1.723s
user   0m1.872s
sys    0m0.048s