Tag Info

Hot answers tagged

22

awk -F, '{sum+=length($3)}; END {print +sum}' file


21

So I went to the source, and it looks like the slowness is in handling double byte characters. Essentially, for every character read in, it needs to call mbrtowc() to try to convert it to a wide character, then that wide character is tested to see if it's a word separator, line separator, etc. Indeed, if I change my locale LANG variable from the default ...


21

cut -d, -f3 | tr -d '\n' | wc -m (remember that wc -c counts bytes, not characters: $ echo a,1,españa,2 | cut -d, -f3 | tr -d '\n' | wc -c 7 $ echo a,1,españa,2 | cut -d, -f3 | tr -d '\n' | wc -m 6 )


19

The command wc aka. word count can do it: $ wc -w <file> example $ cat sample.txt today is a good day $ wc -w sample.txt 5 sample.txt # just the number (thanks to Stephane Chazelas' comment) $ wc -w < sample.txt 5


17

The char type in C is one byte, but it's intended for ASCII characters; there are variable-width encodings like UTF-8 that can take up many bytes per character. wc uses the mbrtowc(3) function to decode multibyte sequences, depending on the locale set by the LC_CTYPE environment variable. If you set the locale properly, you should get the same result for all ...


13

At a guess, Your locale uses UTF-8 encoding, and About 10% of your file consists of characters which require more than one octet to encode into UTF-8. By the way, from man wc: -c, --bytes print the byte counts -m, --chars print the character counts


11

When ls is executed it parses various options. It also detect if output is a tty or not by isatty(). ls.c: code case LS_LS: /* This is for the `ls' program. */ if (isatty (STDOUT_FILENO)) { format = many_per_line; /* See description of qmark_funny_chars, above. */ qmark_funny_chars = true; } else { format = ...


10

In your first example echo will add it's newline at the end, you can stop this by adding the -n option to echo. wc counts characters, words and lines, lines are defined as zero or more characters ending in line feed (\n).


8

Both bash and zsh have a >(pipeline) feature: psql ... | tee >(wc -l) | bzip2 Note that the > here is not a normal redirection, but a necessary part of the syntax. You would need a second > if you wanted to combine it with an actual redirection (with a space in between so it wouldn't be read as >> redirect-for-append).


7

Another answer, which doesn't involve awk but gives you the output in another format: $ grep '.*' *.txt A.txt:45 B.txt:35 C.txt:100


7

One way: awk '{ print $0 " " FILENAME }' A.txt B.txt C.txt Output: 45 A.txt 35 B.txt 100 C.txt


7

how about feeding the pattern file back in as a data file so that each pattern finds at least one match, and then subtracting one from the final reported count for each match grep -f patterns.in logfile.txt patterns.in | cut -f2 -d':' | sort | uniq -c | awk '{print($1 - 1" "$2)}'


7

There is no end-of-file character in Unix or Linux filesystems. The read() system call returns 0 on end-of-file condition, if the file descriptor in use refers to a regular file. read() works differently on sockets and pipes. You don't get a special character to mark end of file. wc gave you 30 as a character or byte count because the first line has 12 ...


7

If you're running Linux, your wc probably comes from GNU Coreutils and has a --files0-from option to read a file (or stdin) containing an arbitrarily long list of NUL-terminated names of file to count. The GNU Coreutils wc documentation says "This is useful when the list of file names is so long that it may exceed a command line length limitation. In such ...


7

Your system should have GNU grep, that has an option -P to use Perl expressions and you can use that, combined with -c (so no need for wc -l): grep -Pvc '\S' somefile The '\S' hands the pattern \S to grep and matches all line containing anything that is not space, -v selects all the other lines (those only with space), and -c counts them. From the man ...


7

The reason why you get 3 here has already been explained, but to add a bit more about the question in the subject: By the strict (POSIX) definition of the term, a text line is always terminated by a newline character, so counting the number of newline characters is the same as counting the number of lines. Sometimes though, you find files that have data ...


6

I came up with this for JUST the number: wc -w [file] | cut -d' ' -f1 5 I also like the wc -w < [file] approach


6

One approach would be to make use of ls to give us a list of the files, but we want this list to be guaranteed to show only 1 file or directory per line. The -1 switch will do this for us. $ ls -1 dir1 dir2 dir3 fileA fileB fileC Example Create the above sample data in an empty directory. $ mkdir dir{1..3} $ touch file{A..C} Check it: $ ls dir1 dir2 ...


5

ls ends each filename with a newline (\n) and not a NUL (\0) (if its standard output is not a terminal). A way to list the files in the current directory, using NUL as a separator, is: find . -maxdepth 1 -print0. This will match the files starting with a period too. To ignore them, use: find . -maxdepth 1 \! -name '.*' -print0 Others ways could be: ls ...


5

If your grep is the GNU grep, here is a quick and dirty solution: grep -A1 "Prepare to remove role" | grep "Delete Successful" | wc -l The grep option -A1 tells grep to print the matching line AND one line following the matching line. The second grep then only prints the lines where the delete is successfull. Note that this will only work reliably when ...


5

Your assumption is correct. wc -l counts the number of newlines (\n). As XTian said, echo adds a newline to the end of everything it prints unless you tell it not to with -n. You can check that this is what's going on by piping through od -c: $ echo -e '\n\n' | od -c 0000000 \n \n \n 0000003 So wc -l correctly counts three lines.


5

A perl solution: perl -Mopen=:locale -F, -anle '$sum += length($F[2]); END{print $sum}' file or a shorter version: perl -Mopen=:locale -F, -anle '$sum += length($F[2])}{print $sum' file


4

There is no newline, so wc -l is correct. Instead, you want to count the number of start of lines. One way to do it: $ diff -y --suppress-common-lines a b | grep '^' | wc -l 1


4

With shell only: for f in {A,B,C}.txt; do echo $(<$f) $f; done Output: 45 A.txt 35 B.txt 100 C.txt


4

I suspect that wc will use less CPU time, it is written in C and has been carefully optimized over time. However, there is no reason not to test this yourself. time will let you compare the performance of your perl script against wc. Using the time command, you should get output like: > time wc /usr/share/dict/words 119095 119095 1145922 ...


4

If your goal is only to detect whether there's always the same number of tabs per line (no bash, no awk): sed 's/[^\t]//g' file | sort -u | wc -l If it outputs 1, then it's good! Or, replacing sed with tr: tr -cd \\t\\n < file | sort -u | wc -l or if you like useless uses of cats and don't like concatenating options: cat file | tr -c -d \\t\\n | ...


4

Try this, and apologies for being obvious: cat *.cs | wc -l or, with git: git ls-files -z ${1} | xargs -0 cat | wc -l If you actually want the output to look like wc output, with both individual counts and a sum, you could use awk to add up the individual lines: git ls-files -z ${1} | xargs -0 wc -l | awk ...


4

watch can be used to repatedly run a command. The -n option specifies the time in seconds that the command is repeated. Commands that include pipes need to be quoted $ watch -n1 'wc -l < twitter.stream'


3

With awk: awk '/Delete Successful/ && last_line ~ /Prepare to remove role/ {n++} {last_line=$0} END {print n+0}'


3

Must it be done with wc? Because here I've ran into a very nice attempt to use regex as a csplit pattern. I don't have a system to test it right now but the regex itself seem to do the job. The expression looks like that: csplit input-file.txt '/([\w.,;]+\s+){500}/'



Only top voted, non community-wiki answers of a minimum length are eligible