I need to efficiently count every character of an arbitrary file by its character CLASS (as defined by the BASH man page); i.e.

[[:alnum:]], [[:alpha:]], [[:ascii:]], [[:blank:]], [[:cntrl:]], [[:digit:]], [[:graph:]], [[:lower:]], [[:print:]], [[:punct:]], [[:space:]], [[:upper:]], [[:word:]] and [[:xdigit:]]

Once the file is processed, display on a single line the resulting counts for each, even when zero.

Web searches have not been fruitful in finding something along these lines.

The arbitrary file (/tmp/f1.txt) will contain a variety of diverse text/data.

I am not looking to process ELF binaries nor UniCode (or any form of multi-byte) content.

I am not concerned about line count (CR and/or LF), only fixated on accumulating a count of each 'character' in the target file by the above classes.

I intend for this to end up as a standard function() in a larger bash script. Bash/sed/awk and the like are desired; while perl/python/ruby not so much.

Sample data files could be:

  • Zero bytes, i.e. no content at all.

  • A single character

  • A single word

  • Multiple words separated by whitespace

  • Multiple lines interspersed with whitespace and/or CarriageReturns and/or LineFeeds.

  • For multiline files there might not be a CR or LF to signify the end of the last line (yet all characters should still be counted).

3 Answers 3

for class in alnum alpha blank cntrl digit graph lower print punct space upper xdigit
  printf '%7s: %d\n' "$class" "$(tr -Cd "[:${class}:]" < "$file" | wc -m)"

ascii and word are not standard character classes and are bash specific. word is alnum plus underscore, and ascii is characters 0 to 127, so you can do:

printf '%7s: %d\n' word "$(tr -Cd "_[:alnum:]" < "$file" | wc -m)"
printf '%7s: %d\n' ascii "$(LC_ALL=C tr -cd '\0-\177' < "$file" | wc -c)"

(note that the GNU implementation of tr, as of coreutils-8.22, won't work with multi-byte characters).

On systems using the GNU libc at least, you can also run:

$ locale ctype-class-names

To find out the list of character classes that are defined in your locale.


Sounds like a fun class! What is it?

This will get you most of the way there; sed doesn't seem to support :ascii: or :word:, but:

for f in alnum alpha ascii blank cntrl digit graph lower print punct space upper word xdigit
  echo "$f: $(sed s/[^[:$f:]]//g b.txt | tr -d '\n' | wc -c)"

Where we use sed to strip out everything except the characters we care about, then delete all the blank lines, and just get the count of characters left over.

Should be relatively accurate, with the possible exception of +/-1 or off-by-a-factor-of-ten errors.

  • That doesn't work for cntrl or space since you're excluding the LF characters. Depending on the sed implementation, NUL characters may be an issue as well. Jul 23, 2013 at 8:51

I'm not going to be looking up what characters fit into to which class - probably you can figure that out and/or just consult the other answers. But this will give you an unequivocal representation of your file character by character without missing any:

 _c2o() { od -A n -t o1 -w1 -v | tr -dc '0-9\n' ; } 
 _c2o <file

That's a function I use in a lot of different ways. Each line is a single bite expressed in octal format - though od is very configurable. But in this way you can very easily just grep or sed for your target values and implement a line counter. It's really a piece of cake. And it's very fast.

Ok, so I went ahead and did the classes anyway:

_classes() { set -- ${classes=alnum alpha blank cntrl digit graph lower print punct space upper xdigit}
        while ${1+:} false ; do
                printf %b $(printf '\\%04o\n' $(seq 0 127)) |
                tr -dc "[:${1}:]" | {
                        printf "$1='"
                        printf "'\n"
                } ; shift

Run the above and you'll get output like:


From there, I'd imagine something like:

eval "$(_classes)"
for class in $classes ; do
    eval "$class=\$(_c2o <file | grep -c -F "$class")"

I need to figure that out a little better - but that just about does the whole job.

  • That doesn't answer the question though. As that gives octal representation of bytes in the file, not characters and you end up being further away from getting character class information. May 7, 2014 at 8:39
  • Stephane, thats true - and i have an answer to that, or most of one, but i dozed off while finishing it up before i could post it. Ill have it up very soon.
    – mikeserv
    May 7, 2014 at 8:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.