The documentation of perl's system() function can be found with perldoc -f system. With perl 5.34, I find:
system LIST
system PROGRAM LIST
Does exactly the same thing as exec, except that a fork is
done first and the parent process waits for the child process to
exit. Note that argument processing varies depending on the
number of arguments. If there is more than one argument in LIST,
or if LIST is an array with more than one value, starts the
program given by the first element of the list with arguments
given by the rest of the list. If there is only one scalar
argument, the argument is checked for shell metacharacters, and
if there are any, the entire argument is passed to the system's
command shell for parsing (this is "/bin/sh -c" on Unix
platforms, but varies on other platforms). If there are no shell
metacharacters in the argument, it is split into words and
passed directly to "execvp", which is more efficient.
Here, with system("ls -R /etc/skel/.[^.]*"), you're in the case where:
- one argument is passed
- the argument contains shell metacharacters, namely
[ and *¹ (^ was a metacharacter in the Bourne shell as an alias for | for backward compatibility with the Thompson shell, but it's no longer in modern POSIX sh).
so that will actually be as if you had written:
system({"/bin/sh"} "sh", "-c", "ls -R /etc/skel/.[^.]*");
Which asks sh to interpret that ls -R /etc/skel/.[^.]* shell code in a child process and waits for its termination.
Except ls -R /etc/skel/.[^.]* is not valid POSIX sh code.
If you look at the specification of Pathname Expansion which in turn refers to Patterns Used for Filename Expansion in the 2018 edition of the POSIX specification, and in particular the part about Patterns Matching a Single Character, you'll find:
[
If an open bracket introduces a bracket expression as in XBD RE Bracket Expression, except that the <exclamation-mark> character ( '!' ) shall replace the <circumflex> character ( '^' ) in its role in a non-matching list in the regular expression notation, it shall introduce a pattern bracket expression. A bracket expression starting with an unquoted <circumflex> character produces unspecified results. Otherwise, '[' shall match the character itself.
In other words, to negate a set you use [!x], not [^x], and what [^x] does is unspecified, it could match the same as [!x] or either ^ or x (like with your sh) or anything as far as POSIX is concerned.
So if the behaviour changed for you, it's likely because your sh changed from one that behaved one way in that regard to one that behaved another way.
In the case of dash (the shell used on Debian, derived from NetBSD sh itself derived from the Almquist shell), there are a number of changes that affect or may affect the behaviour.
That fix is not really relevant to your issue but note that, in turn, it introduces more bugs like:
$ string='\' pattern='[\^x]' dash -c 'case $string in ($pattern) echo match; esac'
match
So with dash linked to the GNU libc, there was a short window in between May and November 2020 where ^ would have been recognised as an alias to ! and your 0.5.11+git20200708+dd9ef66-5 happens to land in the middle of it.
The reason why ^ (from regexp) was changed to ! in globs is historical. As seen above ^ (initially that character was an upward arrow in ASCII, not a caret) was a pipe operator in the Thompson shell and Bourne shell, so echo [^x] would have been the same as echo [ | x] in modern sh.
That ^ alias to | was removed in the Korn shell and POSIX prohibits ^ from being treated as a pipe, but the Korn shell didn't change [!x] back to [^x] to try and preserve backward compatibility. Some other shells such as bash or zsh did (or shells like csh that never had a Bourne heritage baggage), so POSIX leaves it unspecified.
So, your code should have been:
ls -R /etc/skel/.[!.]*
To be valid sh syntax. Now there are still more problems with that code:
- I suppose the intention is to list the hidden files and directories (and their contents) other than
. and .. (which some shells still return in their globs though that's almost never desirable), but note that it misses files named ..foo for instance.
- If there's no matching file, you'll get an error that the file called
/etc/skel/.[^.]* doesn't exist.
perl is a much more capable language than sh, and it's also more portable as there's only one implementation, so instead of asking sh to find the hidden files in /etc to pass to ls, you could instead do it in perl:
@hidden_files = grep {!m{/\.\.?\z}} </etc/skel/.*>;
if (@hidden_files) {
system "ls", "-R", @hidden_files;
}
¹ Strictly speaking, space is also a metacharacter in sh, but it's not considered as such in that perl description; if there are no meta characters other than space, perl does the splitting on space by itself rather than calling sh.
shshells on the two systems rather than focusing on Perl. Perl has absolutely nothing to do with the discrepancy, but one shell (bash?) accepts the non-standard[^.]globbing pattern as "not dot" while the other (dash?) treats it as a standard shell would, as "a circumflex or a dot"./bin/shto bash from dashls.