45

I find myself constantly looking up the syntax of

find . -name "FILENAME"  -exec rm {} \;

mainly because I don't see how exactly the -exec part works. What is the meaning of the braces, the backslash and the semicolon? Are there other use cases for that syntax?

  • 5
    What's wrong about the explanation and examples in the man page? Who votes this up despite there doesn't seem to be the slightest bit of research effort? – Philippos Sep 1 '17 at 9:39
  • 9
    @Philippos: I see your point. Please bear in mind that the man pages are a reference, i.e. usefull for those with an understanding of the matter to look up the syntax. For someone new to the topic, they are often to cryptical and formal to be usefull. You will find that the accepted answer is about 10 times as long as the man page entry, and that's for a reason. – Zsolt Szilagy Sep 1 '17 at 9:53
  • 5
    Even the old POSIX man page reads A utility_name or argument containing only the two characters "{}" shall be replaced by the current pathname, which seems to be suffcient to me. Additionaly it has an example with -exec rm {} \;, just like in your question. In my days, there were barely any other resources than the "big grey wall", books of printed man pages (paper was cheeper than storage). So I know that this is sufficient for someone new to the topic. Your last question though is fair to ask here. Unfortunally neither @Kusalananda nor myself have an answer to that. – Philippos Sep 1 '17 at 10:36
  • 1
    Comeon @Philippos. Are you really telling Kusalananda that he did not improve upon the manpage? :-) – Zsolt Szilagy Sep 1 '17 at 14:19
  • 2
    @ZsoltSzilagy I neither told that nor meant that. He did feed you very well, I just think you are old enough to eat by yourself. (-; – Philippos Sep 1 '17 at 16:19
75

This answer comes in the following parts:

  • Basic usage of -exec
  • Using -exec in combination with sh -c
  • Using -exec ... {} +
  • Using -execdir

Basic usage of -exec

The -exec option takes an external utility with optional arguments as its argument and executes it.

If the string {} is present anywhere in the given command, each instance of it will be replaced by the pathname currently being processed (e.g. ./some/path/FILENAME). In most shells, the two characters {} does not need to be quoted.

The command needs to be terminated with a ; for find to know where it ends (as there may be further options afterwards). To protect the ; from the shell, it needs to be quoted as \; or ';', otherwise the shell will see it as the end of the find command.

Example (the \ at the end of the first two lines are just for line continuations):

find . -type f -name '*.txt'      \
   -exec grep -q 'hello' {} ';'   \
   -exec cat {} ';'

This will find all regular files (-type f) whose names matches the pattern *.txt in or below the current directory. It will then test whether the string hello occurs in any of the found files using grep -q (which does not produce any output, just an exit status). For those files that contain the string, cat will be executed to output the contents of the file to the terminal.

Each -exec also acts like a "test" on the pathnames found by find, just like -type and -name does. If the command returns a zero exit status (signifying "success"), the next part of the find command is considered, otherwise the find command continues with the next pathname. This is used in the example above to find files that contain the string hello, but to ignore all other files.

The above example illustrates the two most common use cases of -exec:

  1. As a test to further restrict the search.
  2. To perform some kind of action on the found pathname (usually, but not necessarily, at the end of the find command).

Using -exec in combination with sh -c

The command that -exec can execute is limited to an external utility with optional arguments. To use shell built-ins, functions, conditionals, pipelines, redirections etc. directly with -exec is not possible, unless wrapped in something like a sh -c child shell.

If bash features are required, then use bash -c in place of sh -c.

sh -c runs /bin/sh with a script given on the command line, followed by optional command line arguments to that script.

A simple example of using sh -c by itself, without find:

sh -c 'echo  "You gave me $1, thanks!"' sh "apples"

This passes two arguments to the child shell script:

  1. The string sh. This will be available as $0 inside the script, and if the internal shell outputs an error message, it will prefix it with this string.

  2. The argument apples is available as $1 in the script, and had there been more arguments, then these would have been available as $2, $3 etc. They would also be available in the list "$@" (except for $0 which would not be part of "$@").

This is useful in combination with -exec as it allows us to make arbitrarily complex scripts that acts on the pathnames found by find.

Example: Find all regular files that have a certain filename suffix, and change that filename suffix to some other suffix, where the suffixes are kept in variables:

from=text  #  Find files that have names like something.text
to=txt     #  Change the .text suffix to .txt

find . -type f -name "*.$from" -exec sh -c 'mv "$3" "${3%.$1}.$2"' sh "$from" "$to" {} ';'

Inside the internal script, $1 would be the string text, $2 would be the string txt and $3 would be whatever pathname find has found for us. The parameter expansion ${3%.$1} would take the pathname and remove the suffix .text from it.

Or, using dirname/basename:

find . -type f -name "*.$from" -exec sh -c '
    mv "$3" "$(dirname "$3")/$(basename "$3" ".$1").$2"' sh "$from" "$to" {} ';'

or, with added variables in the internal script:

find . -type f -name "*.$from" -exec sh -c '
    from=$1; to=$2; pathname=$3
    mv "$pathname" "$(dirname "$pathname")/$(basename "$pathname" ".$from").$to"' sh "$from" "$to" {} ';'

Note that in this last variation, the variables from and to in the child shell are distinct from the variables with the same names in the external script.

The above is the correct way of calling an arbitrary complex script from -exec with find. Using find in a loop like

for pathname in $( find ... ); do

is error prone and inelegant (personal opinion). It is splitting filenames on whitespaces, invoking filename globbing, and also forces the shell to expand the complete result of find before even running the first iteration of the loop.

See also:


Using -exec ... {} +

The ; at the end may be replaced by +. This causes find to execute the given command with as many arguments (found pathnames) as possible rather than once for each found pathname. The string {} has to occur just before the + for this to work.

find . -type f -name '*.txt' \
   -exec grep -q 'hello' {} ';' \
   -exec cat {} +

Here, find will collect the resulting pathnames and execute cat on as many of them as possible at once.

find . -type f -name "*.txt" \
   -exec grep -q "hello" {} ';' \
   -exec mv -t /tmp/files_with_hello/ {} +

Likewise here, mv will be executed as few times as possible. This last example requires GNU mv from coreutils (which supports the -t option).

Using -exec sh -c ... {} + is also an efficient way to loop over a set of pathnames with an arbitrarily complex script.

The basics is the same as when using -exec sh -c ... {} ';', but the script now takes a much longer list of arguments. These can be looped over by looping over "$@" inside the script.

Our example from the last section that changes filename suffixes:

from=text  #  Find files that have names like something.text
to=txt     #  Change the .text suffix to .txt

find . -type f -name "*.$from" -exec sh -c '
    from=$1; to=$2
    shift 2  # remove the first two arguments from the list
             # because in this case these are *not* pathnames
             # given to us by find
    for pathname do  # or:  for pathname in "$@"; do
        mv "$pathname" "${pathname%.$from}.$to"
    done' sh "$from" "$to" {} +

Using -execdir

There is also -execdir (implemented by most find variants, but not a standard option).

This works like -exec with the difference that the given shell command is executed with the directory of the found pathname as its current working directory and that {} will contain the basename of the found pathname without its path (but GNU find will still prefix the basename with ./, while BSD find will not do that).

Example:

find . -type f -name '*.txt' \
    -execdir mv {} done-texts/{}.done \;

This will move each found *.txt-file to a pre-existing done-texts subdirectory in the same directory as where the file was found. The file will also be renamed by adding the suffix .done to it.

This would be a bit trickier to do with -exec as we would have to get the basename of the found file out of {} to form the new name of the file. We also need the directory name from {} to locate the done-texts directory properly.

With -execdir, some things like these becomes easier.

The corresponding operation using -exec instead of -execdir would have to employ a child shell:

find . -type f -name '*.txt' -exec sh -c '
    for name do
        mv "$name" "$( dirname "$name" )/done-texts/$( basename "$name" ).done"
    done' sh {} +

or,

find . -type f -name '*.txt' -exec sh -c '
    for name do
        mv "$name" "${name%/*}/done-texts/${name##*/}.done"
    done' sh {} +
  • 7
    -exec takes a program and arguments and runs it; some shell commands consist only of a program and arguments but many do not. A shell command can include redirection and piping; -exec cannot (although the whole find can be redirected). A shell command can use ; && if etc; -exec cannot, although -a -o can do some. A shell command can be an alias or shell function, or builtin; -exec cannot. A shell command can expand vars; -exec cannot (although the outer shell that runs the find can). A shell command can substitute $(command) differently each time; -exec cannot. ... – dave_thompson_085 Sep 1 '17 at 16:02
  • ... A shell command can glob, -exec cannot -- although find can iterate over files the same way most globs would, so that's rarely wanted. – dave_thompson_085 Sep 1 '17 at 16:04
  • @dave_thompson_085 Of course, the shell command can be sh itself, which is fully capably of doing all those things – Tavian Barnes Sep 1 '17 at 16:04
  • 2
    Saying it's a shell command is wrong here, find -exec cmd arg \; doesn't invoke a shell to interpret a shell command line, it runs execlp("cmd", "arg") directly, not execlp("sh", "-c", "cmd arg") (for which the shell would end up doing the equivalent of execlp("cmd", "arg") if cmd was not builtin). – Stéphane Chazelas Sep 26 '17 at 11:03
  • 2
    You could clarify that all the find arguments after -exec and up to ; or + make up the command to execute along with its arguments, with each instance of a {} argument replaced with the current file (with ;), and {} as the last argument before + replaced with a list of files as separate arguments (in the {} + case). IOW -exec takes several arguments, terminated by a ; or {} +. – Stéphane Chazelas Sep 26 '17 at 11:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.