Does anybody know a tool to tabulate all the unique unicode characters and their counts in a file?

| improve this question | | | | |

I'm not sure what you mean exactly with "unicode characters". To count the different characters in a file you could do something like this:

$ awk -v FS="" -v OFS="\t" '{for(i=1;i<=NF;i++) char[$i]++} END { for(i in char) print i,char[i]}' input.txt

With -v FS="" we set the field separator to nothing. So each character is handled as a single field. In each line we iterate over these fields using the character as a key for the list and increment the count with ++. If all lines were read, we iterate over the counting list and print each key (which represents the character) and its count.

| improve this answer | | | | |
  • Note that this is going to be locale-dependent (and there are certainly conceptions of "unicode character" that it doesn't satisfy, as you noted). – Michael Homer Jan 22 '19 at 20:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.