1

I am working in CentOS 7 trying to do find / sed one liners to fix a ton of files. Specifically two in a row:

  1. First, to add "ignoreOlderThan = 14d" immediately after every [monitor://...] (working)
  2. Second, to find a [monitor://...] group that has two "ignoreOlderThan" and remove the last occurrence.

I have several hundred files that look similar to this (this is the current test file I'm using):

[default]
host = 10.2.2.15

[monitor://apath]
ignoreOlderThan = 14d
index=test
sourcetype=whatever
ignoreOlderThan = 30d

[monitor://truck]
ignoreOlderThan = 14d

[monitor://apath]
ignoreOlderThan = 14d
index=test
sourcetype=whatever
ignoreOlderThan = 30d

The first full command I use is:

find -name inputs.conf -exec sed -ie 's/\(\[monitor:.*\]\)/\1\nignoreOlderThan = 14d/g' {} +

This one works. It adds ignoreOlderThan = 14d immediately below a [monitor://...].

The second, more complex one, does not work:

find -name inputs.conf -exec sed -ie 's/\(\[monitor[^\]]+\][^\[]?\)\(ignoreOlderThan\s?=\s?[0-9]+\w\)\([^\[]+?ignoreOlderThan\s?=\s?[0-9]+\w\)\([^\[]+\)?/\1\3\4/g' {} +

I've tested several possible scenarios using regex101:

https://regex101.com/r/okCSfl/6

https://regex101.com/r/okCSfl/7

https://regex101.com/r/okCSfl/8

https://regex101.com/r/okCSfl/9

The regex works, so I think the issue is somewhere in the sed command, where I'm a lot weaker. I've escaped the parenthesis as needed for capture groups and the command runs... but it doesn't do anything. I thought it may be because sometimes the 4th capture group doesn't exist, but I've also tested a file where every group would have all 4.

I also read that some sed interprets everything as one line, which is why some of my test cases have no spaces at all between newlines.


Edit: @choroba pointed out that sed does one line at a time and suggested perl and gave an example. I played around a little and got it working with the following:

find -name inputs.conf -exec perl -0777 -pi -e 's/(\[monitor:[^[]+?)^(ignoreOlderThan\s?=\s?[0-9]+\w)([^[]+?^ignoreOlderThan\s?=\s?[0-9]+\w[^[]+)/$1$3/gms' {} +

Demonstrated here:

https://regex101.com/r/okCSfl/10

1
  • sed processes the input line by line. Its regex can't easily match multiple lines.
    – choroba
    May 22, 2019 at 14:12

1 Answer 1

0

sed processes the input line by line. Its regex can't easily match multiple lines.

Perl, on the other hand, can read the whole file when the option -0777 is specified:

perl -0777 -pe 's/^(\[monitor:[^[]+^ignoreOlderThan .*)^ignoreOlderThan = \w+/$1/gms' input > output
  • -0777 slurps the whole file
  • -p prints the input after processing it
  • /g repeats the substitution
  • /s makes . match a newline (which it normally doesn't)
  • /m makes ^ match at the beginning of each newline, not just the whole string (and similarly for $ but we don't need it here)
3
  • Interesting, that's good to know. How would you integrate this into the find portion? I know nothing about perl. The regex looks much different than what I was using. Does $1 equate to \1 in my sed? May 22, 2019 at 14:31
  • You can integrate it similarly to sed. Also, I tried a simple regex, but feel free to try anything else you like. $1 is the first matching group, \1 only works in the pattern itself, e.g. s/(.)\1/$1/.
    – choroba
    May 22, 2019 at 14:35
  • find -name inputs.conf -exec perl -0777 -pi -e 's/([monitor:[^[]+?)^(ignoreOlderThan\s?=\s?[0-9]+\w)([^[]+?^ignoreOlderThan\s?=\s?[0-9]+\w[^[]+)/$1$3/gms' {} + This is what I ended up with May 22, 2019 at 16:17

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .