2

I want to do a global search and replace on a directory of files. There are various ways to do this (many editors, sed+find), but I have an extra requirement. I want, for each possible replacement to run a shell script and only keep the change if the script passes.

As an example, let's imagine I decided I used 'pub' too much in my Rust code, and want to try making my code "less public".

I could run: sed -e 's/pub//g' -i $(find . -name "*.rs") to remove all the occurrences of pub, but some of them are necessary, so I instead want to try removing pubs one at a time, running cargo check after each one.

4
  • I gave a full (simplified) example in my question. Commented May 20, 2022 at 12:53
  • 1
    Something like find . -name "*.rs" -exec sed -i.save 's/pub//g' {}; if yourCheckScript; then rm {}.save; else mv {}.save {}; fi \;?
    – Philippos
    Commented May 20, 2022 at 13:44
  • that only checks if all changes in a file are acceptable, I'd like to check each change in each file one by one. Commented May 20, 2022 at 15:44
  • Can the pattern appear more than once for the same line?
    – thanasisp
    Commented May 20, 2022 at 16:44

1 Answer 1

1

Your description says that you have to test any single replacement and keep it or rollback. So you can do the modification in-place, test and in case of failure restore the backup file, do nothing in case of success.

Also you have to keep track of the occurences, for not testing the same rejected change again. Assuming that the pattern may appear once for a line, you could use something like this for one file:

line_numbers=( $(sed -n '/pub/=' "$1") )

for i in "${line_numbers[@]}"; do
    sed -i.old "${i}s/pub//" "$1" && run_test || mv "$1.old" "$1"
done

We just store the line numbers of occurences so that we test once for every case. The first sed is similar to grep -n to get only the line numbers. If there are no matches, the following loop will run for zero times. The second sed is replacing only for the specific line number $i. Double quotes are used to expand the variable into the sed expression.

And run it for your files:

find . -type f -name '*.rs' -exec sh script.sh {} \;

Also you have to clean up any *.old backup files and of cource select a backup extension not existing into your target directory.

Note that depending on your file structure and the frequency of the pattern and the actual run_test you run, this could be slow because we call many processes. Also always be sure you have a good backup of your data before running anything.

3
  • Thanks, that's great! I didn't know sed could return, and accept, line numbers like that. Commented May 23, 2022 at 8:13
  • 1
    Small request for change (or comment for future readers). It's best to add touch $1 just before done, because if (as I was) you are using make or cargo in run_test, then they will get confused as mv doesn't change the mtime, so the moved back file will look "old". Alternatively use cp instead of mv (but then you also need to rm "$1.old". Commented May 24, 2022 at 15:24
  • I am not aware of your cargo/rust tests but I see you mean that the test finds a file older than the last time the test was executed, after a rejected change. Your cp is very good for what you say! In general, when running something else as the test (independent per change) someone could prefer mv as we want to move the file (more efficient) and also keep the previous mtime.
    – thanasisp
    Commented May 24, 2022 at 16:54

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .