# Perl regex matching apostrophe fails

I'm trying to write a Perl script to transform several sripts into a canonical form. I expected the script

#!/usr/bin/perl -W
use File::Spec;
use Getopt::Long 2.3203 qw(:config auto_help auto_version);
use IO::File;
my  $base = '[A-Z]|\\\w+\{[A-Z]\}'; my$tag  = '[a-z]|\{[A-Z]\+1\}|\$A-Z]+'; my sub = '_(?:tag)'; my sup = '[\']?(?:\^tag)?'; my term = "base(?:sup)?(?:sub)?"; my fbtb = qr/(\\Id)\^\{(term)\}_\{(term)\}/ix; my ft = qr/(\\Id)\^(term)_(term)/ix; my t = qr/(\\Id)_(term)/ix; my btt = qr/(\\Id)_\{(term),(term)\}/ix; my bt = qr/(\\Id)_\{(term)\}/ix; my bpt = qr/(\\Id)_\{(term(?:,term)+)\}/ix; my t1 = "\1\[\{\2\}$";
my  $t2 = "\1$\{\2\},\{\3\}$"; while (my$ifn=shift) {
my $ofn =$ifn.'new';
my ($ifh,$ofh);
open IFH,'<',$ifn or die "Unable to open$ifn for input\n";
open OFH,'>',$ofn or die "Unable to open$ofn for output\n";
while (<IFH>) {
m/(\\Id)_{($base(?:$sup)),($base')/ix and print STDOUT "Match 1$&\n";
m/(\\Id)_{($base)/ix and print STDOUT "Match 2$&\n";
m/(\\Id)_{([A-Z]|\\\w+\{[A-Z]\}\^$tag)/ix and print STDOUT "Match 3$&\n";
s/$fbtb/$t2/ix
and print STDOUT "Changed $&\n"; s/$ft/$t2/ix and print STDOUT "Changed$&\n";
s/$btt/$t2/ix
and print STDOUT "Changed $&\n"; s/$bpt/$t1/ix and print STDOUT "Changed$&\n";
s/$bt/$t1/ix
and print STDOUT "Changed $&\n"; s/$t/$t1/ix and print STDOUT "Changed$&\n";
print OFH;
}
}


to match all of the lines in the data below, but it fails to match an apostrophe followed by a brace.

$\seqname{R} = \Range(\funcseqname{f})$. Then $ID_\seqname{R}$ is a left
$\compose[()]$ identity for $\funcseqname{f}$ and $ID_\seqname{D}$ is a
$\funcname{f} \maps S \to T' \defeq \Id_{T,T'} \compose \funcname{f}$
the inclusion map $\Id_{A',A} \maps A' \hookrightarrow A$ is a
$\Id_{\seqname{S}^i,\seqname{S}^{i+1}}$ is open and continuous, hence
The inclusion map $\Id_{A'} \maps A' \hookrightarrow A$ is continuous.
\Id_{S^1_{\alpha_\beta}, S^2_{\alpha_\beta}}
Then $\ID_{\seqname{S}}$ $\Sigma$-commutes with
and $\ID[{\seqname{S}}] = \ID_{\seqname{S},\seqname{S}}$.
$\Id_{S^1_\alpha,S^2_\alpha} \arin \catname{S}^2_\alpha$ and
$\Id_{S_\alpha,S_\alpha} = \Id[{S_\alpha}] \arin \catname{S}_\alpha$ and
\ID_{(E^1,\seqname{C}^1),(E^2,\seqname{C}^2)},
$\Id_{I.U^2}$ is a morphism of $\seqname{E}^2$. Then define
containing $u^1$. $\Id_{U,\seqname{E}^1}$ is a morphism of
$\catname{E}^1 \subcat[full-] \catname{E}^2$, $\Id_{U,\seqname{E}^1}$ is
is $\Id_{(\seqname{A}^i, \seqname{E}^i, \seqname{C}^i}$.
is $\Id_{(\seqname{A}^i, \seqname{E}^i, \seqname{C}^i}$.
$\Id_{(\seqname{A}^i, \seqname{E}^i, \seqname{C}^i}$.
(\Id_{\pi_1(\seqname{E}^i)}, \Id_{\pi_1(\seqname{C}^i)}),
\Id_{\Functor^\Ck_{\M,\Ck} (\seqname{A}^i, \seqname{E}^i, \seqname{C}^i)}
%\Id[{\seqname{L}^1}] = \Id_{\Functor^{\Ck,\catseqname{M}}_{\Man,\LCS}(\seqname{S}^i)}
\bigl ( \Id[{\Triv{E^i}}], \Id_{\Triv[\Ck-]{C^i}} \bigr ),
$\Id_{(\seqname{A}^i, \seqname{E}^i, \seqname{C}^i}$ is an identity
(\Id_{\Triv{E^i}}, \Id[C^i]),
(\Id_{\Triv{E^i}}, \Id[C^i]),
(\Id_{\Triv{E^i}}, \Id[C^i]),
(\Id_{\Triv{E^i}}, \Id[C^i]),
$\Bun \seqname{B}$ is a category and $Id_{B^\alpha}$ is the identity

• Which regex should match it? Do you mean a left or right brace? – choroba Jul 6 '19 at 20:27
• @choroba As an example, s/$btt/$t2/ix and print STDOUT "Changed $&\n"; should match "the inclusion map$\Id_{A',A} \maps A' \hookrightarrow A$is a". – shmuel Jul 6 '19 at 20:49 • Don't you want to use double quotes when defining$sub and $sup? – choroba Jul 6 '19 at 21:14 • Do you know what "\1" means? It's the chr(1), not $1. – choroba Jul 6 '19 at 21:16

Don't use strings to build regular expressions, use qr//. There's one important difference: when two regular expressions created by qr// are concatenated, each of them is first wrapped in a (?:...). When two strings are concatenated, you just get a string containing the first string followed by the second one. But $base contains a pipe symbol whose range will be restricted by the enclosing (?:...). Cf. my$s = "A',";
my $r = 'A|\{'; print$s =~ $_ ? 1 : 0 for qr/$r,/,      # 1
qr/(?:$r),/; # 0  Also, using a variable containing a variable name doesn't interpolate the second variable in a regex, only one level of interpolation happens: my$r = 'A';
my $s = '$r';
print "A" =~ /$s/; # Doesn't match.  Moreover, \1 in a double quoted string (and replacement in a substitution behaves like a double quoted string) means the chr(1). Perl uses $1 to refer to the first matching group. But again, using a variable containing $1 wouldn't expand it, as only one level of interpolation happens. my$s = 'ABC';
my $r = '$1';
say $s =~ s/(A)BC/$r/r;  # $1  You can use the /ee modifier to force the second interpolation, but then the replacement part must be Perl code syntactically, e.g. my$s = 'ABC';
my $r = '"(" .$1 . ")"';
say $s =~ s/A(B)C/$r/ree;  # (B)

• I've already changed the non-interpolating quoted subpatterns (') to compiled subpatterns (qr//), and I can understand why there would be differences in the treatment of, e.g., \, $, but why is  an issue? Also, Perl doesn't seem to permit me to use qr// for a substitution pattern including \1, \2 and \3; I haven't tried it with$1, $2 and$3. – shmuel Jul 8 '19 at 14:37
• \1 etc. can only be used in the pattern if it corresponds to the first capture group of the same pattern. – choroba Jul 8 '19 at 15:15
• The problem is that the string containing the \1 was not the complete pattern but something that I wanted to interpolate into the pattern. It appears that Perl doesn't support comiled fragments of that sort. – shmuel Jul 8 '19 at 16:40
• It does, but you need to delay the evaluation of the string, e.g. by using /ee`. But be very careful not to run uncontrolled code. See also String::Interpolate. – choroba Jul 9 '19 at 6:51
• /e and /ee don't simply allow interpolation, they change the meaning of the string. – shmuel Jul 10 '19 at 13:59