Today I noticed that something changed in Perl, probably recently, in the way it runs shell commands. Could someone explain what has changed? I cannot find the answer myself and sadly we learned about this change in a hardest way possible. Some new users got an interesting contents of their new home directories...

I'm running a simple command/script:

#!/usr/bin/perl -w

system("ls -R /etc/skel/.[^.]*");

In Debian 11: perl v5.32.1, the output is just the contents of the /etc/skel (as expected):

.  ..  .bash_logout  .bashrc  .face  .face.icon  .kshrc  .profile

But in Debian 12: perl v5.36.0 the globbing ^ is ignored and the whole /etc is read, which means .. is not ignored.

When I change ^ to the alternative symbol !: system("ls -R /etc/skel/.[!.]*");, it runs as expected again.

The question is, what changed in Perl that it treats symbols ! and ^ in system() invocation?

EDIT: 29.09.2023, 19:50

I did some tests on both servers and it looks like something in dash changed?

Debian 11: dash Version: 0.5.11+git20200708+dd9ef66-5 (I do not see a --version flag in dash, so this is from APT).

root@s:~# dash -c 'ls -R /etc/skel/.[^.]*'
/etc/skel/.bash_logout  /etc/skel/.bashrc  /etc/skel/.forward+spam  /etc/skel/.kshrc  /etc/skel/.profile
root@s:~# dash -c 'ls -R /etc/skel/.[!.]*'
/etc/skel/.bash_logout  /etc/skel/.bashrc  /etc/skel/.forward+spam  /etc/skel/.kshrc  /etc/skel/.profile

Debian 12: dash Version: 0.5.12-2

[students] ~ ➽ $ dash -c 'ls -R /etc/skel/.[^.]*' | more

[students] ~ ➽ $ dash -c 'ls -R /etc/skel/.[!.]*'
/etc/skel/.bash_logout  /etc/skel/.bashrc  /etc/skel/.face  /etc/skel/.face.icon  /etc/skel/.kshrc  /etc/skel/.profile

Kind regards, Kamil

  • 7
    You should compare the sh shells on the two systems rather than focusing on Perl. Perl has absolutely nothing to do with the discrepancy, but one shell (bash?) accepts the non-standard [^.] globbing pattern as "not dot" while the other (dash?) treats it as a standard shell would, as "a circumflex or a dot".
    – Kusalananda
    Sep 29 at 16:28
  • 4
    In any Debian 11 or your particular Debian 11 installation? Looks like you switched /bin/sh to bash from dash
    – muru
    Sep 29 at 16:32
  • If all you're really using the scripts for is printing the output for the user to read it's probably harmless, but if you're doing anything to actually process the listed files you should probably try to do the file listing directly in perl with File::Find rather than relying on the behavior of either the shell or ls.
    – Random832
    Oct 2 at 7:28

3 Answers 3


It's not Perl that has changed, it is the default shell on your system. Perl's system() call uses /bin/sh. In recent Debian and Debian-derived systems, that is a symlink to dash, a basic POSIX shell. In older ones, and in many non-debian systems, it is instead a symlink to bash.

And, indeed, the two shells behave differently with [^.]:

$ dash -c 'ls -R /etc/skel/.[^.]*' 2>/dev/null | wc
   2875    2572   45543
$ bash -c 'ls -R /etc/skel/.[^.]*' 2>/dev/null | wc
      5       5     103

You can also easily test this by doing:

$ cd /bin
$ sudo rm sh
$ sudo ln -s bash sh

And then running your Perl script again. You will see it behaves as you expect. Just remember to go back and undo the change:

$ cd /bin
$ sudo rm sh
$ sudo ln -s dash sh
  • 3
    :facepalm: shells change rarely, so I totallly forgot to check, what is the default shell. But I did check now, and in both installations the default is dash (symlink from sh), so the shell did not change. I did some more tests, and it looks like something in dash changed?
    – Kamil
    Sep 29 at 17:49
  • 4
    @Kamil Comparing the sh in the debian:11 docker hub image with that of the docker:12 image, I think this might be a correct assessment. Both are dash, but they behave differently regarding the [!.] v.s. [^.] patterns (the newer version does not accept ^ as an alias for ! in [...]). The missing link is to find the changelog for dash.
    – Kusalananda
    Sep 29 at 17:57
  • 2
    Yes, I'm looking at Debian's changelog now and see some globbing related things recently. metadata.ftp-master.debian.org/changelogs//main/d/dash/…
    – Kamil
    Sep 29 at 18:00
  • 6
    @Kamil Aha, they switch from using the system's fnmatch() implementation (which may treat ^ as ! in [...]) to using a separate implementation of the routine, which presumably is stricter in its conformance to POSIX. In any case, your Perl code should be using ! for maximum portability.
    – Kusalananda
    Sep 29 at 18:05
  • 3
    On Debian or derivatives, changing the /bin/sh symlink would be done with dpkg-reconfigure dash. Sep 29 at 19:59

The documentation of perl's system() function can be found with perldoc -f system. With perl 5.34, I find:

system LIST
Does exactly the same thing as exec, except that a fork is done first and the parent process waits for the child process to exit. Note that argument processing varies depending on the number of arguments. If there is more than one argument in LIST, or if LIST is an array with more than one value, starts the program given by the first element of the list with arguments given by the rest of the list. If there is only one scalar argument, the argument is checked for shell metacharacters, and if there are any, the entire argument is passed to the system's command shell for parsing (this is "/bin/sh -c" on Unix platforms, but varies on other platforms). If there are no shell metacharacters in the argument, it is split into words and passed directly to "execvp", which is more efficient.

Here, with system("ls -R /etc/skel/.[^.]*"), you're in the case where:

  • one argument is passed
  • the argument contains shell metacharacters, namely [ and *¹ (^ was a metacharacter in the Bourne shell as an alias for | for backward compatibility with the Thompson shell, but it's no longer in modern POSIX sh).

so that will actually be as if you had written:

system({"/bin/sh"} "sh", "-c", "ls -R /etc/skel/.[^.]*");

Which asks sh to interpret that ls -R /etc/skel/.[^.]* shell code in a child process and waits for its termination.

Except ls -R /etc/skel/.[^.]* is not valid POSIX sh code.

If you look at the specification of Pathname Expansion which in turn refers to Patterns Used for Filename Expansion in the 2018 edition of the POSIX specification, and in particular the part about Patterns Matching a Single Character, you'll find:

If an open bracket introduces a bracket expression as in XBD RE Bracket Expression, except that the <exclamation-mark> character ( '!' ) shall replace the <circumflex> character ( '^' ) in its role in a non-matching list in the regular expression notation, it shall introduce a pattern bracket expression. A bracket expression starting with an unquoted <circumflex> character produces unspecified results. Otherwise, '[' shall match the character itself.

In other words, to negate a set you use [!x], not [^x], and what [^x] does is unspecified, it could match the same as [!x] or either ^ or x (like with your sh) or anything as far as POSIX is concerned.

So if the behaviour changed for you, it's likely because your sh changed from one that behaved one way in that regard to one that behaved another way.

In the case of dash (the shell used on Debian, derived from NetBSD sh itself derived from the Almquist shell), there are a number of changes that affect or may affect the behaviour.

That fix is not really relevant to your issue but note that, in turn, it introduces more bugs like:

$ string='\' pattern='[\^x]' dash -c 'case $string in ($pattern) echo match; esac'

So with dash linked to the GNU libc, there was a short window in between May and November 2020 where ^ would have been recognised as an alias to ! and your 0.5.11+git20200708+dd9ef66-5 happens to land in the middle of it.

The reason why ^ (from regexp) was changed to ! in globs is historical. As seen above ^ (initially that character was an upward arrow in ASCII, not a caret) was a pipe operator in the Thompson shell and Bourne shell, so echo [^x] would have been the same as echo [ | x] in modern sh.

That ^ alias to | was removed in the Korn shell and POSIX prohibits ^ from being treated as a pipe, but the Korn shell didn't change [!x] back to [^x] to try and preserve backward compatibility. Some other shells such as bash or zsh did (or shells like csh that never had a Bourne heritage baggage), so POSIX leaves it unspecified.

So, your code should have been:

ls -R /etc/skel/.[!.]*

To be valid sh syntax. Now there are still more problems with that code:

  • I suppose the intention is to list the hidden files and directories (and their contents) other than . and .. (which some shells still return in their globs though that's almost never desirable), but note that it misses files named ..foo for instance.
  • If there's no matching file, you'll get an error that the file called /etc/skel/.[^.]* doesn't exist.

perl is a much more capable language than sh, and it's also more portable as there's only one implementation, so instead of asking sh to find the hidden files in /etc to pass to ls, you could instead do it in perl:

@hidden_files = grep {!m{/\.\.?\z}} </etc/skel/.*>;
if (@hidden_files) {
  system "ls", "-R", @hidden_files;

¹ Strictly speaking, space is also a metacharacter in sh, but it's not considered as such in that perl description; if there are no meta characters other than space, perl does the splitting on space by itself rather than calling sh.

  • Hmm, I would consider your asnwer as a definitive one, very detailed and informative -- and valid. I inherited the scirpts, which are much more complicated than the example I provided (it was just to illustrate the problem). They did work for years, despite not being POSIX compliant, but I see some discussion on the forumst and mailing lists, that ^ shoud/shouldn't be keep as a valid negation symbol. The scripts apparently need some work, or probably should be rewritten in Python (people in our environment tend to use Python).
    – Kamil
    Sep 29 at 18:21
  • And I must admit that I saw a lot of ^ in the scripts and on the Internet, and in the Bash manual and up to this point I too considered this as thing that "just works" :)
    – Kamil
    Sep 29 at 18:22
  • @Kamil If it were me, I would merely rebuild perl and this time tell Configure to use /bin/bash for the system shell rather than to use /bin/sh given that the latter points elsewhither now.
    – tchrist
    Oct 1 at 15:20

Nothing. Those symbols are interpreted by your shell, not by Perl.

What system() does is spawn /bin/sh -c with the entire command string as the argument. The shell is what interprets everything else inside that string – that's why it's called a shell command.

Unlike regular expressions (regex), [^abc] is not actually a standard syntax element in shell wildcards (globs) and writing it as [!abc] is the proper way. It just happens that some shells, such as Bash, accept both forms – but /bin/sh is not guaranteed to be Bash or to support any Bash-specific extensions; it is only required to support what POSIX requires out of a shell.

So on Debian, /bin/sh is nowadays much more likely to be linked to dash, a much more minimal shell (optimized for performance), although old installations may still have it linked to Bash as that used to be the default many releases ago. One of the differences is that dash does not support the alternative ^ symbol, only !.

(I also faintly remember something from last month about even Bash 5.2 behaving the same way when invoked "POSIX shell" mode? I can't remember now.)

If I might add, that's really not a good way of listing files through Perl. It has its own glob() function already! And if you want it to be recursive, use the standard File::Find module (or make a recursive Perl function). Even with system(), find would have avoided this issue as it doesn't need the .. exclusion.

  • Yes, this is tnot the best way and just a simple example illustrating the problem (the real script is much more complicated).
    – Kamil
    Sep 29 at 17:50

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .