What am I trying to do?

Write a command that looks for any files that aren't being required by any other file in my project.

What have I tried?

I'm using find to get a list of filenames. Now I want to use the file names as the search string (rather than the file to search).

I'm using find server/lib -type f -exec basename {} \; | cut -f 1 -d '.' to get my list of filenames and stripping away the file extension.

Now I know that this grep -R --exclude-dir=node_modules <search-string> . -l is what I want to pipe each filename into but I don't know how to pass them as the <search-string> argument.

Other info

I'm also vaguely aware that piping to this grep might pass the whole result of the find (the entire list of files) which I obviously don't want to do so if that is what I'm about to do then tell me.

Also, (although this probably isn't in the scope of this question) as I'm using this as a way to find any of the filenames that grep fails to find anything for, any advice on getting this sort of output would be greatly appreciated

> <the-command>

server/lib/foo.js # where these are the list of files in which it appears

Not Found! #where `filename2` wasn't found anywhere (obviously)

Short answer:

If you just want to pass a bunch of lines as arguments to another command, xargs is your friend - in this case, because you're putting it into the middle of a command, you'll want to use the -I {} flag. This sets {} as a placeholder so that you can put wherever (you can set your own placeholder, but I usually stick to {} since it won't be mistaken for many other things.

Putting it all together (assuming the commands you've given do what you think they do!):

find server/lib -type f -exec basename {} \; | cut -f 1 -d '.' | xargs -I {} grep -R --exclude-dir=node_modules {} . -l

Long answer

Ok, so we can probably take this further. Let's say you wanted the format you mentioned earlier - you can xargs to sh -c so you can chain commands:

find server/lib -type f -exec basename {} \; | cut -f 1 -d '.' | xargs -I {} sh -c "echo {}; grep -R --exclude-dir=node_modules {} . -l"

Running this on my machine, it looks like this is giving you the files where the file is found. My brain is tired, so instead of wrapping it in a nice script (which is what you should actually do when oneliners start to get gnarly, and use https://www.shellcheck.net/ while you're at it) you can have this horrible hack instead, which (I think) will give you the output you're looking for:

find server/lib -type f -exec basename {} \; | cut -f 1 -d '.' | xargs -I {} sh -c "echo {}; grep -R --exclude-dir=node_modules {} . -l && printf \"\n\" || printf \"Not found\n\n\""

  • Great thanks, yes xargs -I {} is what I was looking for. – Simon Legg Mar 28 '17 at 9:53

With zsh:

grep -rFl --exclude-dir=node_modules -e${(u)^projects} .
  • **/ recursive globbing. A feature introduced by zsh in the early 90s.
  • (D.:t:r) is a glob qualifier, a zsh-specific feature that allows selecting files using attributes other than name, or change the sorting or do modifications on the found files.
  • D: include dot-files and traverse dot-dirs as find would. Most likely, you'd want to take that one off.
  • .: only regular files (like -type f in find)
  • :t, :r: history modifiers like in csh/bash but here applied to the globbed files. :t for the tail (basename), :r for the root name (extension removed).
  • x${^array} distributes the elements like rc's x^$array or fish's x$array would do. For instance if $array is (1, 2). That becomes x1 x2.
  • ${(u)array} (for unique), remove duplicates ((u) being a parameter expansion flag).

For the list of files that contain none of the strings, replace -l with -L (assuming GNU grep, but you're already using GNU-specific options here). -F is for fixed string search (as you probably don't want those project names to be treated as regexps). You may also want to throw in a -w option to grep for word match so that foo doesn't match in foobar for instance (would still match in foo-bar though).

  • Awesome thanks for this and the explainations, I wasn't aware of the projects=(server/lib/**/*(D.:t:r)) producing an array like that in zsh. However I now need to list the files where grep failed to find anything. I guess I'd need to catch a 0/false exit code? – Simon Legg Mar 26 '17 at 20:22
  • @Simon, see the part about -L at the end of the answer – Stéphane Chazelas Mar 26 '17 at 20:33
  • Yes, I saw that. -l tells me where the search result was and -L tells me where it wasn't found. But I want to know a list of the search-terms that weren't found anywhere rather than where they were/weren't found – Simon Legg Mar 26 '17 at 20:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.