Need help with Perl regex commenting out lines in html

Question

I am trying to comment out a line item entry in a bunch of html documents.

I need to match a pattern that has a few variations and the substitution will the be matched pattern surrounded by <!-- -->

The variations are as follows:

<li><a href="latest-news.htm" >Latest News</a></li>

<li><a href="pages/latest-news.htm" >Latest News</a></li>

<li><a href="../../latest-news.htm"  >Latest News</a>
      </li>

Those are the main variations but there can also be some where latest-news.php instead of .htm. Another variation is only one set of ../

I have this so far:

     find ./ -name "*.htm" -exec perl -p0e 's/(^\s*<li><a href="((\.\.\/)*|pages\/)?latest-news\.(htm|php)"\s*>Latest News<\/a>\s*(\n)?\s*<\/li>\s*)/<!-- $1 -->/g' {} \; | grep -C 1 "latest-news" | grep -C 1 "latest-news"

I don't know what's going wrong there but I'm not matching anything. The grep at the end should show me each instance of this line in each file found and they are not getting commented out.

I'm just starting to use perl and regex and this is one of the most complex attempts I've made so far.

Is what I'm trying to do possible or is there a better way to accomplish this?

I've tried using some regex testing sites but most do not have a standard perl regex test and the ones using PHP regex and match all the variations do not work when I try to run it.

Please let me know if I can add any more information.

I was able to get some of the multi line variations matched a couple of weeks ago but I cannot figure out which version of my command did that. I am currently starting over from a backup of all the files so I can test more.

my versions

$> perl -v

This is perl 5, version 28, subversion 1 (v5.28.1) built for x86_64-linux-gnu-thread-multi
(with 61 registered patches, see perl -V for more detail)

$> grep -V
grep (GNU grep) 3.3

$> uname -mrs
Linux 5.3.0-24-generic x86_64

$> lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 19.10
Release:    19.10
Codename:   eoan

UPDATE

I updated my regex search to:

(<li><a href="(\.\.\/|pages\/)*latest-news.(htm|php)"\s*>Latest News<\/a>\s*(\n)?\s*<\/li>)\n?

This caught all but a few of the single line variations. Some were still not matched if they didn't have ../ or page/ before latest-news.htm. There is an instance of a line that has more white space (3 tabs vs 1 tab or space characters) before the <li> that wasn't matched.

If I add \s* before the <li> in my search pattern the comment starts at the end of the previous line but still does not catch the line with more white space in the beginning.

            <li><a href="latest-news.htm" class="current">Latest News</a></li>

^ This is the line that still won't match.

Answer 1

Maybe something like:

perl -0777 -pe '
   s{<li>\s*<a\s[^>]*href="[^"]*latest-news\.(?:htm|php)"[^>]*>\s*Latest News\s*</a>\s*</li>}
    {<!-- $& -->}g' your-file.htm

would be enough.

Note that if you don't enable the multi-line mode (with the m flag), ^ only matches at the start of the subject (each NUL-delimited record with -0, the whole input with -0777), not the start of any line in the subject. Also note that \s matches on newline as well. Use \h if you only want to match on horizontal spacing (but AFAIK HTML makes no difference between horizontal and vertical, NL and SPC are interchangeable as far as its syntax is concerned at least outside of things like <pre>, CDATA..., quoted attribute values..).

To avoid doing the substitution inside already commented sections, you could do:

perl -0777 -pe '
   s{(<!--.*?-->)|<li>\s*<a\s[^>]*href="[^"]*latest-news\.(?:htm|php)"[^>]*>\s*Latest News\s*</a>\s*</li>}
    {$1 // "<!-- $& -->"}gse' your-file.htm