I need to perform a global (repeated) replace operation on the text of files recursively in a directory, but there are a few constraints that have given me issues:
- I need to replace in a way that includes a newline character, which breaks sed, as mentioned here: Why is sed giving me an error about an unterminated `s'?
- I need to output the files in a different directory because I'm using AWS DataPipeline and need to work around an issue that was occurring due to making changes to the files from the InputDirectory during the command execution. (So, I need to write each modified file to a file with the same filename but located in the OutputDirectory.)
- I also ran into issues with trying to glob the output filename
So, I'm left with something that perhaps would:
- use
find
to enumerate the files - loop through the files and for each file, replace the input path with the output path (perhaps even with
sed
) cat
the file and pipe the contents toawk
and write the contents to the updated path.- construct any directories needed in the process (which I believe should happen automatically from output redirection, but I want to say it explicitly just in case)
Is there a straightforward way to do this?
\n
for newline in the replacement string.globstar
?\n
in the replacement string is not portable! We don't know whether devinbost or future readers use GNUsed
. Better use a newline escaped by a backslash.\n
breaks when I run it in AWS DataPipeline, as per the link that I included in the post.