4

Brace expansions can produce multiple instances of file names if there is an overlap in the matches.

A simple example:

mkdir testdir; cd testdir
touch abcd
for f in *{b,c}*; do something_to "$f"; done
# or more simply:
ls -l *{b,c}*

That is a very simplified example just for illustration. In this case, the single file abcd would be processed (or listed) twice.

How can this list be best deduplicated?

  • An associative array could be used.
  • A carefully crafted glob/brace expansion could be used, but that's not robust.
  • zsh has the same behavior. – jordanm Mar 10 '16 at 17:18
  • @jordanm: Thanks, How's it going? – Dennis Williamson Mar 10 '16 at 17:34
  • What's not robust is combining shell globs with brace expansion. Shell globs by themselves are perfectly robust, you just have to know what you're doing. – Wildcard Mar 11 '16 at 5:30
  • @Wildcard: There's nothing wrong with using globs and brace expansion together. Believe me, I know what I'm doing. – Dennis Williamson Mar 11 '16 at 18:57
5

Yes,

for f in *{b,c}*

is first expanded to:

for f in *b* *c*

And then the loop runs over the independent expansion of those two globs.

What you want here is one glob. Here, *[bc]* would do, but for anything more complex, in bash, you'd need to enable ksh-like extended globs. You'll probably want the nullglob option as well:

shopt -s nullglob extglob
for f in *@(b|c)*; do...

In zsh:

for f in *(b|c)*(N); do

The (N) being for a per-glob nullglob.

In ksh93:

for f in ~(N)*@(b|c)*; do
  • The key to the deduplication is the vertical bar or operator whereas the brace expansion is implicitly an and operation which just begs for duplication. – Dennis Williamson Mar 10 '16 at 23:09

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.