5

I have a command that reads a file, and outputs a modified version on stdout:

./convert /path/to/file

How can I recursively apply this command to all files in a directory, and overwrite the existing contents of each file with the result from the command above?

I found this question which is very similar, but all solutions offered involve outputting the results to a single file, which is not what I want.

1

4 Answers 4

6

If I understand correctly, you can convert one file with

./convert /path/to/file >/path/to/file.new
mv /path/to/file.new /path/to/file

To apply a command to every file in a directory tree, use the find utility. Since you need to execute a complex command for each file, you need to invoke a shell explicitly.

find /path/to/top/directory -type f -exec sh -c '
  /path/to/convert "$0" >"$0.new" &&
  mv "$0.new" "$0"
' {} \;
4
  • Although correct enough, the {}parameter is not necessary. Per the find man page, we have "All following arguments to find are taken to be arguments to the command until an argument consisting of ; is encountered. The string {} is replaced by the current file name being processed .. ". So, the final set of brackets is converted to the current filename being processed and passed to sh, which apparently disregards it. Consequentially, we can replace all instances of $0 as {} and get the same result, if you like.
    – sherrellbc
    Jan 6, 2016 at 16:45
  • 1
    @sherrellbc No, sh does not disregard its argument. It is available in the script as $0, and the script needs it, as you can see. Replacing $0 by {} doesn't work in general because some find implementations don't recognize {} if it's a substring, and even on those that do, this fails on file names containing characters that have a special meaning in the shell (which ones depends on how you quote the {} but there's no combination of quotes that works for all file names). Jan 6, 2016 at 17:00
  • Considering what OP requested it would not simply be on your code second line /path/to/convert "$0" ? I suppose the OP script overwrites the file to be converted. Thank you Jan 9, 2016 at 18:56
  • 1
    @joao_pimentel /path/to/convert writes the modified version to stdout, so this content needs to be redirected to the output file. Since the output file is the same as the input file, it needs to be written to a temporary file (path/to/convert "$0" >"$0" would overwrite the input file before the program has read it). Jan 9, 2016 at 18:59
2

The question you linked to does not operate recursively, it works only one level deep. For recursion, use find:

find . -type f -exec /path/to/convert {} \;

You want to move convert itself out of the tree you're walking for risk of it trying to modify itself.

6
  • Please read the question carefully, I want to write the output of my script to the file in question, overwriting its existing contents. Because convert only outputs the modified file.
    – BenMorel
    Oct 15, 2013 at 15:19
  • @Benjamin How does convert operate? Is it reading the file line by line or does it slurp the whole thing into memory first?
    – Joseph R.
    Oct 15, 2013 at 15:31
  • It slurps the whole thing in memory first. TBH I'm realizing that my simplest option is to rewrite the script to allow outputting to a file instead of stdout, but I'm still curious to know whether my original request is achievable!
    – BenMorel
    Oct 15, 2013 at 15:35
  • @Benjamin I see. And I suppose you ruled out redirecting stdout for fear of truncating your file if the command gets interrupted, correct?
    – Joseph R.
    Oct 15, 2013 at 15:38
  • Not necessarily, no! I guess I just don't know the proper syntax to do it.
    – BenMorel
    Oct 15, 2013 at 16:17
2

If I understand correctly, you want to overwrite each input file with the result of convert on that input file. If so, try this:

find . -type f | while IFS= read -r file; do
  ./convert "$file" > /tmp/foo.tmp && mv /tmp/foo.tmp "$file";
done
0

Borrowing from @Gilles' excellent answer:

find /path/to/top/directory -type f -exec sh -c '/path/to/convert "$0" | tee "$0"' {} \;

I tried the above with sed, it worked.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .