Complex regex sed replacement not working but not throwing errors

Question

I am working in CentOS 7 trying to do find / sed one liners to fix a ton of files. Specifically two in a row:

1. First, to add "ignoreOlderThan = 14d" immediately after every [monitor://...] (working)
2. Second, to find a [monitor://...] group that has two "ignoreOlderThan" and remove the last occurrence.

I have several hundred files that look similar to this (this is the current test file I'm using):

[default]
host = 10.2.2.15

[monitor://apath]
ignoreOlderThan = 14d
index=test
sourcetype=whatever
ignoreOlderThan = 30d

[monitor://truck]
ignoreOlderThan = 14d

[monitor://apath]
ignoreOlderThan = 14d
index=test
sourcetype=whatever
ignoreOlderThan = 30d

The first full command I use is:

find -name inputs.conf -exec sed -ie 's/\(\[monitor:.*\]\)/\1\nignoreOlderThan = 14d/g' {} +

This one works. It adds ignoreOlderThan = 14d immediately below a [monitor://...].

The second, more complex one, does not work:

find -name inputs.conf -exec sed -ie 's/\(\[monitor[^\]]+\][^\[]?\)\(ignoreOlderThan\s?=\s?[0-9]+\w\)\([^\[]+?ignoreOlderThan\s?=\s?[0-9]+\w\)\([^\[]+\)?/\1\3\4/g' {} +

I've tested several possible scenarios using regex101:

https://regex101.com/r/okCSfl/6

https://regex101.com/r/okCSfl/7

https://regex101.com/r/okCSfl/8

https://regex101.com/r/okCSfl/9

The regex works, so I think the issue is somewhere in the sed command, where I'm a lot weaker. I've escaped the parenthesis as needed for capture groups and the command runs... but it doesn't do anything. I thought it may be because sometimes the 4th capture group doesn't exist, but I've also tested a file where every group would have all 4.

I also read that some sed interprets everything as one line, which is why some of my test cases have no spaces at all between newlines.

---

Edit: @choroba pointed out that sed does one line at a time and suggested perl and gave an example. I played around a little and got it working with the following:

find -name inputs.conf -exec perl -0777 -pi -e 's/(\[monitor:[^[]+?)^(ignoreOlderThan\s?=\s?[0-9]+\w)([^[]+?^ignoreOlderThan\s?=\s?[0-9]+\w[^[]+)/$1$3/gms' {} +

Demonstrated here:

https://regex101.com/r/okCSfl/10

Answer 1

sed processes the input line by line. Its regex can't easily match multiple lines.

Perl, on the other hand, can read the whole file when the option -0777 is specified:

perl -0777 -pe 's/^(\[monitor:[^[]+^ignoreOlderThan .*)^ignoreOlderThan = \w+/$1/gms' input > output

• -0777 slurps the whole file
• -p prints the input after processing it
• /g repeats the substitution
• /s makes . match a newline (which it normally doesn't)
• /m makes ^ match at the beginning of each newline, not just the whole string (and similarly for $ but we don't need it here)