2

I would like to understand how to correctly send the output of a find + exec command to a pipeline for further processing.

i.e. how can we
(1) select a group files
(2) perform some operation on the group via exec and
(3) use the output of that operation as input to 1 or more filters in a pipeline

For example, when I try to filter a find + exec command like so, I see lots of 'terminated by signal 13' errors for the lines that I filtered out.

$ find c* -name "*.jpg" -exec ls {} \; | head
c0/1467058201899.jpg
c0/1465854461118.jpg
c0/1465855196637.jpg
c0/1467050962421.jpg
c0/1465856476225.jpg
c0/1467050385287.jpg
c0/1465853696999.jpg
c0/1467144293032.jpg
c0/1467051637981.jpg
c0/1465841226352.jpg
find: `ls' terminated by signal 13
find: `ls' terminated by signal 13
find: `ls' terminated by signal 13
...

I can make this particular error go away like so, but this does not feel very elegant.

$ find c* -name "*.jpg" -exec ls {} \; -print 2>/dev/null | head
c0/1467058201899.jpg
c0/1467058201899.jpg
c0/1465854461118.jpg
c0/1465854461118.jpg
c0/1465855196637.jpg
c0/1465855196637.jpg
c0/1467050962421.jpg
c0/1467050962421.jpg
c0/1465856476225.jpg
c0/1465856476225.jpg

Is there a more elegant way to do this for the general case of find + exec where the command being executed may vary?

UPDATE
using xargs still seems generate output to stderr ...

$ find c* -name "*.jpg" -print0 | xargs -0 ls | head
c0/1465425913832.jpg
c0/1465425968779.jpg
c0/1465426112741.jpg
c0/1465426116540.jpg
c0/1465426121623.jpg
c0/1465426127656.jpg
c0/1465426133584.jpg
c0/1465426140097.jpg
c0/1465426143185.jpg
c0/1465426156715.jpg
xargs: ls: terminated by signal 13

using find + exec terminating with + instead of ; also generates output to stderr ...

$ find c* -name "*.jpg" -exec ls {} \+ | head
c0/1465425913832.jpg
c0/1465425968779.jpg
c0/1465426112741.jpg
c0/1465426116540.jpg
c0/1465426121623.jpg
c0/1465426127656.jpg
c0/1465426133584.jpg
c0/1465426140097.jpg
c0/1465426143185.jpg
c0/1465426156715.jpg
find: `ls' terminated by signal 13
find: `ls' terminated by signal 13

though adding "-print 2>/dev/null" to this command results in a command that executes very quickly ...

$ find c* -name "*.jpg" -exec ls {} \+ -print 2>/dev/null | head
c0/1467058201899.jpg
c0/1465854461118.jpg
c0/1465855196637.jpg
c0/1467050962421.jpg
c0/1465856476225.jpg
c0/1467050385287.jpg
c0/1465853696999.jpg
c0/1467144293032.jpg
c0/1467051637981.jpg
c0/1465841226352.jpg
  • 1
    There are of course many other ways you can achieve the same thing, but your "not very elegant" solution works fine. In general, when using "find" to pipe files to some command (other than 'ls'), I use 'xargs', e.g. "find /etc -type f -name '*.jpg' -mtime +30 | xargs rm". This is much more efficient than using the find command's "ls" option, which forks a process for each and every match. – Lee-Man Jul 12 '16 at 22:12
  • 1
    @Lee-Man You should use find -print0 and xargs -0. By default, these utilities print and split on newlines, which means if I can put a file in /etc, I could name it /etc/simplefile\n/home/leeman/mission-critical-document, and rm would get /etc/simplefile as one argument, and /home/leeman/mission-critical-document as another. This is because newlines are allowed in filenames, but NULs are not. – Dylan Frese Jul 12 '16 at 22:18
  • 3
    Or with modern find, no need for xargs: find ... -exec ... {} + – thrig Jul 12 '16 at 22:26
  • 2
    Why are you using ls at all? If you just did a -print in the find rather than the more obscure and much less efficient -exec ls {} then find would know you had stopped listening and exit gracefully. – MAP Jul 13 '16 at 3:17
  • Why are you using ls? find will produce a list by itself... – Kusalananda Jul 13 '16 at 18:49
3

The error is occurring because find doesn't know when to stop. If you run find | head, when head gets its ten lines and exits, the next time find tries to write a filename, it'll get a SIGPIPE (letting it know that the other end of the pipe is broken or closed), and find will gracefully exit.

But here, find isn't writing anything, ls is. find can see that its children are dying, and it can see why, but it doesn't know that the other end of the pipe is closed, and it doesn't know to stop spawning them.

  • I don't understand what you're trying to say in your third paragraph. find definitely does not capture the output of commands that it executes for -exec, that output goes wherever it goes (here, to the pipe). – Gilles 'SO- stop being evil' Jul 12 '16 at 23:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.