8

I want to output all lines containing the word OK recursively from a directory. But there are a few extensions which I need to exclude from the result:

*~
*.map
*.js except *.debug.js

I tried:

grep -r --exclude={*~,*.map} "OK" /some/dir

Except that I don't know how to remove from the result all those non-debug .js files.

7

I would just pass that through a second grep to remove them:

grep -r --exclude={\*~,\*.map} "OK" bar/ | grep -vP '(?<!debug)\.js'

The -v reverses the match, printing lines that don't match the pattern and the -P enables Perl Compatible Regular Expressions which let us use negative lookbehinds. This particular regex, will match .js that is not prececeded by debug which means (since we are inverting the matches) that only those .js files will be printed.

However, as @QuestionOverflow pointed out int the comments, that could have the unintended side effect of filtering out lines that contain OK and js since the grep -v is applied to the entire output, not only the file name. To avoid that, just add a colon (that's what grep uses to separate file names from file contents):

grep -r --exclude={*~,*.map} "OK" bar/ | grep -vP '(?<!debug).js:'

That will still fail if your input line contains foo.js: or if your file name contains :. So, to be sure, use a different approach:

grep -Tr --exclude={*~,*.map} "OK" bar/ | grep -vP '(?<!debug).js\t'

The -T causes grep to print a tab between the file name and the file contents. So, if we simply add a \t to the end of the regex, it will only match against file names, and not the contents of the line.

Still, using find might make more sense regardless.

  • 1
    Would I be inadvertently excluding lines in those files that I want, but containing both OK and .js on the same line? – Question Overflow Apr 16 '14 at 9:06
  • @QuestionOverflow ah, yes indeed, good catch. See updated answer. – terdon Apr 16 '14 at 9:20
  • Fantastic answer. Have to accept yours since I ask specifically for grep. Thanks. – Question Overflow Apr 16 '14 at 9:26
  • @QuestionOverflow you're very welcome. In general though, find is probably better for this kind of thing. Getting the right grep can be tricky as you pointed out :). – terdon Apr 16 '14 at 9:27
  • Your solutions fail if one has the failglob option set in the shell: bash: no match: --exclude=*~ You need to quote your GLOB pattern arguments to --exclude to hide them from shell expansion, e.g. --exclude={\*~,\*.map} – Ian D. Allen Apr 16 '14 at 15:28
7

I'd use find to locate the files and pipe the result through xargs:

$ find . -type f \! -name "*~" \
                 \! -name "*.map" \
                 \! \( -name "*.js" -and \! -name "*.debug.js" \) \
         -print0 | xargs -0 grep "OK"

This searches for every file not matching "*~", "*.map" or "*.js but not *.debug.js".

Using find you can easily search for rather complex rules and this approach saves you from accidentally removing false positives as could happen with double grep.

  • Nice answer too :) – Question Overflow Apr 16 '14 at 9:27
  • 3
    Yes, this is probably the best way, +1. You could also use -exec grep OK {} + instead of xargs and avoid an extra program. – terdon Apr 16 '14 at 9:33
  • 2
    @IDAllen no, note that I suggested -exec + not -exec \;, that will run as few commands as possible, much like xargs. – terdon Apr 16 '14 at 17:20
4

With zsh you can do:

setopt extendedglob
grep OK some/dir/**/^(*~|*.map|(^*debug).js)

Provided of course the argument list isn't too long, in which case you can always do:

printf '%s\0' some/dir/**/^(*~|*.map|(^*debug).js) | xargs -0 grep OK
  • Also, you could make the last one zsh-only: autoload zargs and zargs some/dir/**/^(*~|*.map|(^*debug).js) -- grep OK – don_crissti Dec 12 '15 at 1:11
2

If you don't mind seeing the output slightly out of order (if you do, you can sort it):

grep -r --exclude={*~,*.map,*.js} "OK" /some/dir **/*.debug.js

This requires that your shell supports ** for recursive globbing: zsh does out of the box, bash does after you run shopt -s globstar, ksh93 does after you run set -o globstar.

Without ** support in the shell, you can use two grep commands:

grep -r --exclude={*~,*.map,*.js} "OK" /some/dir
grep -r --include=*.debug.js "OK" /some/dir
  • My shell supports **, but there seem to be something wrong with the extra argument **/*.debug.js, causing grep to interpret OK as a directory. Have you tried running it? – Question Overflow Apr 17 '14 at 3:50
  • @QuestionOverflow My mistake, I'd swapped the order of the arguments. – Gilles Apr 17 '14 at 7:36
1

You can use ripgrep. By default it ignores hidden files and respects your .gitignore file.

You can specify the inclusion or exclusion rules by using the following parameters:

-g/--glob GLOB Include or exclude files and directories for searching that match the given glob.

-t/--type TYPE Only search files matching TYPE. Multiple type flags may be provided.

-T/--type-not TYPE Do not search files matching TYPE.

Use the --type-list flag to list all available types.

Here are few simple examples:

rg -Tjs "OK"                              # Excludes *.js, *.jsx, *.vue files.
rg -tpy "OK"                              # Includes Python files.
rg --type-add 'map:*.map' -tmap PATTERN   # Excludes *.map files.
rg -g '!*.js' -g '*.debug.js' PATTERN     # Excludes *.js apart of *.debug.js.

Here is the complete solution to exclude *.~, *.map, *.js, but not *.debug.js:

rg -g '*.*' -g '!*.~' -g '!*.map' -g '!*.js' -g '*.debug.js' "OK"

Testing:

$ touch file.~ file.map file.js file.debug.js file.txt file.md
$ rg --files
file.debug.js
file.js
file.map
file.md
file.txt
$ rg -g '*.*' -g '!*.~' -g '!*.map' -g '!*.js' -g '*.debug.js' --files
file.debug.js
file.md
file.txt

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.