Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

As a C programmer, I was surprised to see that wc -c (which count the number of bytes), and wc -m (which counts the number of characters) output very different results for a long, text file of mine. I had always been told that sizeof(char) is 1 byte.

qdii@nomada ~/Documents $ wc -c sentences.csv
102990983 sentences.csv
qdii@nomada ~/Documents $ wc -m sentences.csv
89023123 sentences.csv

Any explanation?

share|improve this question
    
See @rici's answer below... you've got your -m and -c flags backwards in your question (c = bytes, m = characters)... your example output is correct, though. –  Dan Oct 16 '12 at 19:16
    
@Dan thanks, this is fixed –  qdii Oct 16 '12 at 19:51
add comment

2 Answers 2

up vote 17 down vote accepted

The char type in C is one byte, but it's intended for ASCII characters; there are variable-width encodings like UTF-8 that can take up many bytes per character. wc uses the mbrtowc(3) function to decode multibyte sequences, depending on the locale set by the LC_CTYPE environment variable. If you set the locale properly, you should get the same result for all cases. For example:

qdii@nomada ~/Documents $ LC_CTYPE="C" wc -m sentences.csv
102990983 sentences.csv
share|improve this answer
add comment

At a guess,

  1. Your locale uses UTF-8 encoding, and

  2. About 10% of your file consists of characters which require more than one octet to encode into UTF-8.

By the way, from man wc:

   -c, --bytes
          print the byte counts

   -m, --chars
          print the character counts
share|improve this answer
    
Thanks for adding correction regarding -m & -c flags. –  Dan Oct 16 '12 at 19:15
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.