Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I was wondering how to count the number of a specific character in each line by some text processing utilities?

For example, to count " in each line of the following text

"hello!" 
Thank you!

The first line has two, and the second line has 0.

Another example is to count ( in each line.

share|improve this question
    
Just going to add that you received much increased performance by writing your own 10 line C program for this rather than using regular expressions with sed. You should consider doing depending on the size of your input files. –  user606723 Aug 14 '11 at 22:23

10 Answers 10

up vote 30 down vote accepted

You can do it with sed and awk:

$ sed 's/[^"]//g' dat | awk '{ print length }'
2
0

Where dat is your example text, sed deletes (for each line) all non-" characters and awk prints for each line its size (i.e. length is equivalent to length($0), where $0 denotes the current line).

For another character you just have to change the sed expression. For example for ( to:

's/[^(]//g'

Update: sed is kind of overkill for the task - tr is sufficient. An equivalent solution with tr is:

$ tr -d -c '"\n' < dat | awk '{ print length; }'

Meaning that tr deletes all characters which are not (-c means complement) in the character set "\n.

share|improve this answer
    
+1, but { print length } is shorter, and is equivalent to length of whole line (length($0)). –  enzotib Aug 14 '11 at 19:09
    
@enzotib, yes, you are right - somehow I thought that $0 would also include the newline character - which is not the case. I will update the answer. –  maxschlepzig Aug 14 '11 at 19:14
1  
+1 should be more efficient than the tr&wc version. –  Stéphane Gimenez Aug 14 '11 at 19:41
1  
Yes, but can it handle Unicode? –  amphetamachine Aug 15 '11 at 10:51
    
@amphetamachine, yes - at least a quick test with ß (utf hex: c3 9f) (instead of ") works as expected, i.e. tr, sed and awk do complement/replacement/counting without a problem - on a Ubuntu 10.04 system. –  maxschlepzig Aug 15 '11 at 18:29

I would just use awk

awk -F\" '{print NF-1}' <fileName>

Here we set the field separator (with the -F flag) to be the character " then all we do is print number of fields NF - 1. The number of occurrences of the target character will be one less than the number of separated fields.

For funny characters that are interpreted by the shell you just need to make sure you escape them otherwise the command line will try and interpret them. So for both " and ) you need to escape the field separator (with \).

share|improve this answer
    
Maybe edit your answer to use singles quotes instead for escaping. It will work with any character (except '). Also, it has a strange behavior with empty lines. –  Stéphane Gimenez Aug 15 '11 at 16:08
    
The question specifically uses " so I feel obliged to make the code work with it. It depends what shell you are using weather the character needs to be escaped but bash/tcsh will both need to escape " –  Loki Astari Aug 15 '11 at 16:10
    
Of course, but there is no problem with -F'"'. –  Stéphane Gimenez Aug 15 '11 at 16:12
    
+1 What a good idea to use FS.... This will resolve the blank-line showing -1, and, for example, the "$1" from the bash commandline. ... awk -F"$1" '{print NF==0?NF:NF-1}' filename –  Peter.O Aug 15 '11 at 22:19

Using tr ard wc:

function countchar()
{
    while read -r i; do echo "$i" | tr -dc "$1" | wc -c; done
}

Usage:

$ countchar '"' <file.txt  #returns one count per line of file.txt
1
3
0

$ countchar ')'           #will count parenthesis from stdin
$ countchar '0123456789'  #will count numbers from stdin
share|improve this answer
3  
Note. tr doesn't handle characters which use more than one byte.. see Wikipedia tr (Unix) .. ie. tr is not Unicode compliant. –  Peter.O Aug 15 '11 at 19:43

Yet another implementation that do not relies on external programs, in bash or ksh:

while IFS= read -r line; do 
  line="${line//[^\"]/}"
  echo ${#line}
done <input-file

Use line="${line//[^(]}"for counting (.

share|improve this answer
    
When the last line doesn't have a trailing \n, the while loop exits, because although it read the last line, it also returns a non-zero exit code to indicate EOF... to get around it, the following snippet works (..It has been been bugging me for a while, and I've just discovered this workaroung)... eof=false; IFS=; until $eof; do read -r || eof=true; echo "$REPLY"; done –  Peter.O Aug 15 '11 at 21:42
    
@Gilles: you added a trailing / that is not needed in bash. It is a ksh requirement? –  enzotib Aug 16 '11 at 7:35
1  
The trailing / is needed in older versions of ksh, and IIRC in older versions of bash as well. –  Gilles Aug 16 '11 at 8:15

I decided to write a C program cause I was bored.

You should probably add input validation, but other than that's all set.

#include <stdio.h>
#include <string.h>

int main(int argc, char *argv[])
{
        char c = argv[1][0];
        char * line = NULL;
        size_t len = 0;
        while (getline(&line, &len, stdin) != -1)
        {
                int count = 0;
                char * s = line;
                while (*s) if(*s++ == c) count++;
                printf("%d\n",count);
        }
        if(line) free(line);
}
share|improve this answer
    
Thanks! Thanks for being bored so that I can learn something. Oh wait, do you need a return? –  Tim Aug 14 '11 at 23:31
    
*shrugs*, if you want to be fully correct, you also need to add a few more #includes, but the default warnings on my compiler doesn't seem to care. –  user606723 Aug 14 '11 at 23:39
    
You can leave out the free(line) because exiting the program implicitly frees all allocated memory - then there is place for a return 0; ... ;). Even in examples it is not good style to leave the return code undefined. Btw, getline is a GNU extension - in case someone is wondering. –  maxschlepzig Aug 15 '11 at 6:04
1  
@Tim, yes, e.g. if you refactor the code such that it is a standalone function - say -f, which is called several times from other code, then you have to call free after the last call of getline at the end of this function f. –  maxschlepzig Aug 15 '11 at 7:44
1  
There's not enough JQuery in this answer anyone? i.stack.imgur.com/ssRUr.gif ;) –  Bob Apr 30 '13 at 13:19

Another possible implementation with awk and gsub:

awk '{ gsub("[^\"]", ""); print length }' input-file

The function gsub is the equivalent of sed's 's///g' .

Use gsub("[^(]", "")for counting (.

share|improve this answer
    
You can save one character, i.e. when removing the stdin redirection ... ;) –  maxschlepzig Aug 14 '11 at 20:34
    
@maxschlepzig: yeah, of course ;) –  enzotib Aug 14 '11 at 20:43
    
awk '{print gsub(/"/,"")}' input-file would be enough, as "For each substring matching the regular expression r in the string t, substitute the string s, and return the number of substitutions." (man awk) –  manatwork Sep 6 '11 at 12:42

Here is another C solution that only needs STD C and less memory:

#include <stdio.h>

int main(int argc, char **argv)
{
  if (argc < 2 || !*argv[1]) {
    puts("Argument missing.");
    return 1;
  }
  char c = *argv[1], x = 0;
  size_t count = 0;
  while ((x = getc(stdin)) != EOF)
    if (x == '\n') {
      printf("%zd\n", count);
      count = 0;
    } else if (x == c)
      ++count;
  return 0;
}
share|improve this answer
    
This will not report on the last line if it doesn't have a trailing '\n' –  Peter.O Aug 15 '11 at 22:24
    
@fred, yes, which is on purpose, because a line without a trailing \n is no real line. This is the same behavior as with my other sed/awk (tr/awk) answer. –  maxschlepzig Aug 16 '11 at 7:25

The answers using awk fail if the number of matches is too large (which happens to be my situation). For the answer from loki-astari, the following error is reported:

awk -F" '{print NF-1}' foo.txt 
awk: program limit exceeded: maximum number of fields size=32767
    FILENAME="foo.txt" FNR=1 NR=1

For the answer from enzotib (and the equivalent from manatwork), a segmentation fault occurs:

awk '{ gsub("[^\"]", ""); print length }' foo.txt
Segmentation fault

The sed solution by maxschlepzig works correctly, but is slow (timings below).

Some solutions not yet suggested here. First, using grep:

grep -o \" foo.txt | wc -w

And using perl:

perl -ne '$x+=s/\"//g; END {print "$x\n"}' foo.txt

Here are some timings for a few of the solutions (ordered slowest to fastest); I limited things to one-liners here. 'foo.txt' is a file with one line and one long string which contains 84922 matches.

## sed solution by [maxschlepzig]
$ time sed 's/[^"]//g' foo.txt | awk '{ print length }'
84922
real    0m1.207s
user    0m1.192s
sys     0m0.008s

## using grep
$ time grep -o \" foo.txt | wc -w
84922
real    0m0.109s
user    0m0.100s
sys     0m0.012s

## using perl
$ time perl -ne '$x+=s/\"//g; END {print "$x\n"}' foo.txt
84922
real    0m0.034s
user    0m0.028s
sys     0m0.004s

## the winner: updated tr solution by [maxschlepzig]
$ time tr -d -c '\"\n' < foo.txt |  awk '{ print length }'
84922
real    0m0.016s
user    0m0.012s
sys     0m0.004s
share|improve this answer

If $x is the variable containing your string:

expr length "${x//[^\"]/}"

The ${x// thing removes all chars except ", expr length then counts the characters of the resulting string.

share|improve this answer

To check that all the lines are the same (if, for example, you are validating a CSV file), you might want to add the following to the end of your command, to get a list of unique counts (should give only 1 value):

| sort -u
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.