2

I have the large set of the text file. In that, each article is separated by 15 stopwords. I want to find out the total number of words count in that file excluding the stopword

  • 3
    try this: sed 's/stopword//g' file | wc -w – taliezin Nov 3 '17 at 12:49
  • 1
    seems like a reasonable Answer to me, @taliezin! – Jeff Schaller Nov 3 '17 at 13:08
  • 2
    It would help (for accurate answers) to see what your input looks like— a small representative sample. – Jeff Schaller Nov 3 '17 at 13:14
  • 1
    What defines a word? – 123 Nov 3 '17 at 13:17
5

With GNU grep:

grep -Eo '\S+' < file | grep -vcxF stopword

Would count (-c) the number of words (with the same definition of word as that of wc -w, at least on valid text, that is sequences of non-spacing characters (\S+)) that are not (-v) exactly (-xF) stopword.

3

The number of words in input minus the number of stopwords (using GNU grep's -o, since you tagged Linux):

echo $(( $(wc -w < input) - $( grep -o stopword input | wc -l ) ))

Sample input:

I have the large set of the text file. In that, each article is separated by 15 stopwords. I want to find out the total number of words count in that file excluding the stopword.
stopword stopword stopword stopword stopword stopword stopword stopword stopword stopword stopword stopword stopword stopword stopword
I have the large set of the text file. In that, each article is separated by 15 stopwords. I want to find out the total number of words count in that file excluding the stopword.

Output:

$ echo $(( $(wc -w < input) - $( grep -o stopword input | wc -l ) ))
66
  • 1
    Note that on an input like stopwordstopword-stopword, that would give -2 – Stéphane Chazelas Nov 3 '17 at 13:11
2
awk '{ gsub("stopword",""); words+=NF }; END { print words; }' /text/file

This counts everything that awk considers a field. Even it is semantically not a word like

  • a hyphen
  • a dot after a space (wrong end of sentence . Next sentence)
  • numbers in headings ( 1. introduction )

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.