19

I am looking for a command to count number of all words in a file. For instance if a file is like this,

today is a 
good day

then it should print 5, since there are 5 words there.

2
43

The command wc aka. word count can do it:

$ wc -w <file>

example

$ cat sample.txt
today is a 
good day


$ wc -w sample.txt
5 sample.txt


# just the number (thanks to Stephane Chazelas' comment)
$ wc -w < sample.txt
5
1
  • 1
    Note that words for wc -w don't have the same definition as for GNU grep -w. For wc a word is a sequence of one or more non-space characters ([:space:] character class in the current locale). For instance foo,bar and foo bar (with a non-breaking space) are each one word. Jun 12 '14 at 15:18
7

I came up with this for JUST the number:

wc -w [file] | cut -d' ' -f1

5

I also like the wc -w < [file] approach

Finally, for storing just the word count in a variable, you could use the following:

myVar=($(wc -w /path/to/file))

This lets you skip the filename elegantly.

1
3

The better solution is using Perl:

perl -nle '$word += scalar(split(/\s+/, $_)); END{print $word}' filename

@Bernhard

You can check the source code of wc command from coreutils, I test in my machine, with file subst.c in bash 4.2 source.

time wc -w subst.c

real    0m0.025s
user    0m0.016s
sys     0m0.000s

And

time perl -nle '$word += scalar(split(" ", $_)); END{print $word}' subst.c

real    0m0.021s
user    0m0.016s
sys     0m0.004s

The bigger the file is, the more efficient Perl is with respect to wc.

16
  • 13
    Why is this better than wc?
    – Sparr
    Jun 19 '13 at 17:13
  • 2
    @Sparr for one thing because, to my very great surprise, it seems to be much faster. I tried it on a text file with 141813504 words and wc took ~14sec while Perl took ~5sec!
    – terdon
    Jun 19 '13 at 17:28
  • 3
    I think the 'bigger' issue really is an answer that has a dependency on Perl and I'm never a big fan of such a dependency. If the question was about performance that would be another thing. Jun 19 '13 at 17:31
  • 5
    Note that a split on /\s+/ is like a split(' ') except that any leading whitespace produces a null first field. That difference will give you one extra word (the null first field, that is) per line link. So use (split(" ", $_)) otherwise for a file created like this: echo -e "unix\n linux" > testfile your one-liner reports 3 words. Jun 19 '13 at 18:03
  • 1
    Your timings show that wc is quicker (it's user and sys times that matter there). With LC_ALL=C, wc will be significantly quicker, just like with PERLIO=:utf8, perl will be significantly slower. Jun 28 '13 at 10:20
3

Let's use AWK!

$ function wordfrequency() { awk 'BEGIN { FS="[^a-zA-Z]+" } { for (i=1; i<=NF; i++) { word = tolower($i) words[word]++ } } END { for (w in words) printf("%3d %s\n", words[w], w) } ' | sort -rn } 
$ cat your_file.txt | wordfrequency

This lists the frequency of each word occurring in the provided file. I know it's not what you asked for, but it's better! If you want to see the occurrences of your word, you can just do this:

$ cat your_file.txt | wordfrequency | grep yourword

I even added this function to my .dotfiles


Source: AWK-ward Ruby

1
  • It counts words, so it is good enough for me! :-)
    – aggsol
    Jun 18 '18 at 6:57
3

The wc program counts "words", but those are not for instance the "words" that many people would see when they examine a file. The vi program for instance uses a different measure of "words", delimiting them based on their character classes, while wc simply counts things separated by whitespace. The two measures can be radically different. Consider this example:

first,second

vi sees three words (first and second as well as the comma separating them), while wc sees one (there is no whitespace on that line). There are many ways to count words, some are less useful than others.

While Perl would be better suited to writing a counter for the vi-style words, here is a quick example using sed, tr and wc (moderately portable using literal carriage returns ^M):

#!/bin/sh
in_words="[[:alnum:]_]"
in_punct="[][{}\\|:\"';<>,./?\`~!@#$%^&*()+=-]"
sed     -e "s/\($in_words\)\($in_punct\)/\1^M\2/g" \
        -e "s/\($in_punct\)\($in_words\)/\1^M\2/g" \
        -e "s/[[:space:]]/^M/g" \
        "$@" |
tr '\r' '\n' |
sed     -e '/^$/d' |
wc      -l

Comparing counts:

  • Running the script on itself, gives me 76 words.
  • The example in Perl by @cuonglm gives 31.
  • Using wc gives 28.

For reference, POSIX vi says:

In the POSIX locale, vi shall recognize five kinds of words:

  1. A maximal sequence of letters, digits, and underscores, delimited at both ends by:

    • Characters other than letters, digits, or underscores

    • The beginning or end of a line

    • The beginning or end of the edit buffer

  2. A maximal sequence of characters other than letters, digits, underscores, or characters, delimited at both ends by:

    • A letter, digit, underscore
    • <blank> characters
    • The beginning or end of a line
    • The beginning or end of the edit buffer
  3. One or more sequential blank lines

  4. The first character in the edit buffer

  5. The last non-<newline> in the edit buffer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.