BASH: count # of words in each line of a document

Question

I need to identify patterns in a text file for further analysis. So the input files may contain semi-structured text as follows;

file1

905:john: abc123: [email protected]: US  
920:eric: ericaA: [email protected]: US  
1000: rio: ri0ri0: [email protected]: IN  

file2

nathen <tab> [email protected] <tab> 764323545 <tab> UK
thomas <tab> [email protected] <tab> 563363421 <tab> UK
ian <tab> [email protected] <tab> 3453245472 <tab> SP

number of words in a line may vary for each document. delimiter also vary but unique for each document. what I want is to count number of words per each line in each document.

output would be:

for file1

5  
5 
5 
5   

for file2

4  
4 
4 
4

I want generalize this for any file with any delimiter. It could be - | : \space+ \tab+.

Some files are as follows:

| [email protected] | er34532 |  
| [email protected] | 764474 |

Answer 1

For file2 with its simple format of white-space separated words, try:

$ awk '{print NF}' file2
4
4
4

In awk, NF is the number of fields. In awk by default, words characters separated by white-space.

For file1, for which words may be separated by colons or spaces, we need to add : to the field separator:

$ awk -F'[: \t]+' '{print NF}' file1
5
5
5

Improvement

awk will think there are extra fields if a line has leading or trailing whitespace. We can fix that by removing any such whitespace before we count the fields, like this for file1:

awk -F'[: \t]+' '{gsub(/^[: \t]+|[: \t]+$/, ""); print NF}' file1

or this for file2:

awk '{gsub(/^[[:space:]]+|[[:space:]]+$/, ""); print NF}' file2

In the regex, ^[[:space:]]+ matches leading whitespace and [[:space:]]+$ matches trailing whitespace. Their combination in the gsub command removes both.

Generalization

If the field separator could be any of - | : or , then use:

awk -F'[-|: \t]+' '{gsub(/^[[-\|: \t]+|[[-\|: \t]+$/, ""); print NF}'

Example:

$ cat file3
| [email protected] | er34532 |  
| [email protected] | 764474 |
$ awk -F'[-|: \t]+' '{gsub(/^[[-\|: \t]+|[[-\|: \t]+$/, ""); print NF}' file3
2
2

Answer 2

You could use this trick in perl to count substrings matching a word regex of your choosing e.g. to count sequences of "words" consisting of perl's \w character set augmented with @ and .

perl -lne 'print my $count = () = $_ =~ /[\w@.]+/g' somefile

which gives

$ perl -lne 'print my $count = () = $_ =~ /[\w@.]+/g' file1
5
5
5

and

$ perl -lne 'print my $count = () = $_ =~ /[\w@.]+/g' file2
4
4
4

for your sample files.