shopt -s extglob not working as expected

Question

I am trying to recursively change file permissions in a wordpress installation affecting everything below the public_html document root folder, excluding certain folders.

I have set extglob on with shopt -s extglob

When I run the following command it fails to exclude the folder I want it to exclude ls -R -alh ./public_html/!(*uploads*) | grep uploads

The above lists numerous files in the uploads folder -- it should not list any.

Of course the command above is only a test; I'll be using chmod in the actual script.

I have tried numerous different combinations and all have failed; for example, the following:

ls -R -alh ./public_html/!(wp-content/uploads*)

ls: cannot access ./public_html/!(wp-content/uploads*): No such file or directory

This is my first attempt to use exclusions with shell commands and I might be making a simple mistake.

Any ideas what I am doing wrong?

Please note: the purpose of the question is to better understand extglob rather than find alternatives. I can script alternatives but as I have never used extglob before I am having trouble understanding the syntax.

Answer 1

First, the extglob controls what ls sees on its command line. It does not control what ls does with what it sees on the command line. This is important because the -R option to ls tells ls to explore recursively any directories it sees on the command line. So, even if the *uploads* directories are not given explicitly on the command line, ls will find them when it explores their parent directories.

Second, as you know, don't parse ls. The output of ls is not meant for use in pipelines or scripts. Trying to use it that way eventually leads to unhappiness.

Third, to get the files that you want, try:

find ./public_html ! -path '*uploads*'

To explain:

• The ./public_html tells find to start looking in the ./public_html directory.

• By itself, the option -path '*uploads*' matches on any path that contains the pattern *uploads*. (-path is similar to find's -name option but path includes the directory names.) The preceding !, however, indicates negation. So, the option ! -path '*uploads*' excludes any path matching *uploads*.

To get ls style output while still using the features of find, consider:

find ./public_html ! -path '*uploads*' -exec ls -dalh {} +

Answer 2

John1024 has added a good solution to your apparent problem, but based on the comments, you'd like to understand extglob better. I suspect that one of the core misunderstandings may have been that extglobs would work in a recursive manner -- that saying ./public_html/!(*uploads*) would exclude something named ./public_html/wp-content/uploads/. The globs (filename expansion) only work on a single directory level at a time. Buried in the bash manual, under Filename expansion is:

When matching a filename, the slash character must always be matched explicitly.

So, as John1024 said, the ls command did not have any (expanded) parameters to it that were of the form ./public_html/*uploads*, yet it could have had a parameter such as ./public_html/wp-content (which doesn't match the excluded pattern), which it was then told -R to recursively list.

In order to exclude *uploads* directories using extglobs at the command-line, you'd have to build up a command that had as many exclusions as there are subdirectories:

ls -dalh ./public_html/!(*uploads*) \
         ./public_html/!(*uploads*)/!(*uploads*) \
         ./public_html/!(*uploads*)/!(*uploads*)/!(*uploads*)
# ... etc ...

... being also careful not to use ls's -R flag. The same -R risk would apply to a chmod command. A potential risk here is that if any subdirectory has "too many" files, you risk filling the argument list for ls. This approach is fragile, in that it will not catch newly-created directories that are deeper than the number of patterns you've written, and it would break (ls will complain of a missing directory) if you ever had fewer subdirectories than the number of given patterns, which is the long explanation for why find is the right tool for this job.