0

I recently read How to find the total number of occurrences of text and files with find command, which asked about a way to find the number of times text "abc" appears in a file. The answer there provided the command find . -name "*.txt" | xargs grep -i "abc" | wc -l to find the count.

Previously I have used something like more "file_name*" |grep "abc" |wc -l to list the number if times text "abc" appears in a specific file(s).

I tried this and found that the more command returned faster, but seemed to use ~30% more CPU (I was also monitoring via top) than the find command.

I was wondering if anyone had some more solid data on which of these two commands would be more intensive on resources if you were to query through approximately 15 files? How about 30+ files?

Improve this question
5
  • @manatwork Yes, I ran one command in the same directory on the same files, then I ran the other right after it finished.
    – SirCobalt
    May 4, 2012 at 15:30
  • If you have a fast hard disk and multi-core CPU, you can make the find statement faster (but it will burn your CPU and hard disk bandwidth resources). Example of xargs running 4 concurrent process for the grep command: find . -print0 -name "*.txt" | xargs -0 -P 4 grep -i "abc" | wc -l The combination of -print0 and xargs -0 helps with filenames containing spaces!
    – Huygens
    May 4, 2012 at 15:44
  • Are you aware that when more displays multiple files, it inserts a header with the file name before each file's content? So if one of your files' name contains the string you search for, greping in more's output will find extra occurrences in the file names. To avoid this, better use cat.
    – manatwork
    May 4, 2012 at 15:48
  • Thanks @Huygens! I don't believe any of the files I was searching through had spaces in the name (simply because files with spaces almost always cause some sort of issue), but thank you for the input! The xargs -P 4 is what will create 4 separate instances of the command to run?
    – SirCobalt
    May 4, 2012 at 15:48
  • Take such comparisons with caution! The system has a cache, the HDD has a cache - at least perform the test multiple times, and in opposite order.
    – user unknown
    May 4, 2012 at 20:33

3 Answers 3

Reset to default
5

You don't need more (or cat) if you have the list of file you need to grep: just give grep the files as an argument (no need to pipe the data through a second tool):

grep -i abc *.txt | wc -l

The main difference is that find will not only list the files in the current directory (as the shell expansion of *.txt) but it will recurse into subdirectories too:

find . -name "*.txt" | xargs grep -i abc | wc -l

In the second case if there are some *.txt files in subdirectories these are also supplied as arguments to the grep command.

And by the way grep has a -c option to count the occurrences (no need for wc either)

grep -c -i abc *txt

also it will just give the number of occurrences per file and not the total

Improve this answer
1
  • This is good to know as well! The way I was initially taught to search through files was by using more and piping in grep. I've never used grep as much of a standalone command.
    – SirCobalt
    May 4, 2012 at 15:36
1

I don't have "solid data", but consider this.

more is a (primitive) filter for paging through text one screenful at a time. It's intent is to be used interactively "for crt viewing". So even though you are sending the output to a pipe, it is still using the memory and CPU resources to display the each file to you with the filtering features more offers.

It would be more correct in your example to use cat rather than more. Taken a step further, your example command can be made even more efficient by dropping the additional step of going through a filtering program and have grep directly search through the files (instead of piping the output to it).

In regards to the find command being slower, find is also traversing the directory structure starting from . whereas more is only working on "file_name*" in the current working directory.

Improve this answer
3
  • Thank you for the information! I never really knew the difference from commands such as more and cat, so this is very helpful.
    – SirCobalt
    May 4, 2012 at 15:32
  • No cat is not more correct, it is as useless as more. You can just give the files to process as arguments to grep. grep abc *.txt. There is no need to involve an additional too.
    – Matteo
    May 4, 2012 at 15:33
  • @Matteo Good point.
    – George M
    May 4, 2012 at 15:38
1

It only looks at first sight like a attempt for the useless use of cat award, but if we cat all files, before invoking grep:

cat *.txt | grep -ci abc

grep counts the sum for you. Since you like to traverse subdirectories (you like to, don't you?), you can do it there too:

find -name "*.txt" -exec cat {} + | grep -ci abc
Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .