5

I am required to write a one-time util that does some operation on all the files in a given directory but a list of files in a predefined list. Since the given list is predefined, I am going to hard-code it as an array.

Having said that, how to get names of all files that are not in the given array? This can be in any standard unix script(bash, awk, perl).

6

With bash, you could do:

all=(*)
except=(file1 file2 notme.txt)
only=()
IFS=/
for file in "${all[@]}"; do
  case "/${except[*]}/" in
    (*"/$file/"*) ;;     # do nothing (exclude)
    (*) only+=("$file")  # add to the array
  esac
done
ls -ld -- "${only[@]}"

(that works here for the files in the current directory, but not reliably for globs like all=(*/*) except=(foo/bar) as we use / to join the elements of the array for the look-up).

It's based on the fact that "${array[*]}" joins the elements of the array with the first character of $IFS (here chosen to be / as it can't otherwise occur in a file name; NUL is a character that can't occur in a file path, but unfortunately bash (contrary to zsh) can't have such a character in its variables). So for each file in $all (here with $file being foo as an example), we do a case "/file1/file2/notme.txt/" in (*"/foo/"*) to check if $file is to be excluded.

  • Hi @Stéphane, That works awesome. I am just trying to find out explantion for the statements using *. It would be helpful If you add it. – Kannan Ramamoorthy Aug 10 '17 at 12:38
  • I need details about 2 more fact using . The statement ("/$i/") and () only+=("$i") . I am also parallelly trying to find it as well. – Kannan Ramamoorthy Aug 10 '17 at 14:48
  • @Kannan, is it clearer after this new edit? – Stéphane Chazelas Aug 10 '17 at 14:53
  • any reason not to use find with ! -name foos (followed by an appropriate -exec)? – Kevin Aug 10 '17 at 16:54
  • @Kevin, that's getting away from the original question. Also, -name takes a wildcard pattern, not file name, so you'd need to escape the wildcard operators. – Stéphane Chazelas Aug 10 '17 at 16:58
5

It's easier with zsh:

except=(file1 file2 notme.txt)
all=(*)
only=(${all:|except})
ls -ld -- $only

Mnemonic for ${all:|except}: elements of $all bar those of $except.

You can also check if files are in the $except array as part of a glob qualifier:

ls -ld -- *.txt(^e:'((except[(Ie)$REPLY]))':)

Or using a function:

in_except() ((except[(Ie)${1-$REPLY}]))
ls -ld -- *.txt(^+in_except)
3

If the filenames are simple enough, you could use bash's GLOBIGNORE variable:

The GLOBIGNORE shell variable may be used to restrict the set of filenames matching a pattern. If GLOBIGNORE is set, each matching filename that also matches one of the patterns in GLOBIGNORE is removed from the list of matches. If the nocaseglob option is set, the matching against the patterns in GLOBIGNORE is performed without regard to case. The filenames . and .. are always ignored when GLOBIGNORE is set and not null. However, setting GLOBIGNORE to a non-null value has the effect of enabling the dotglob shell option, so all other filenames beginning with a ‘.’ will match. To get the old behavior of ignoring filenames beginning with a ‘.’, make ‘.*’ one of the patterns in GLOBIGNORE. The dotglob option is disabled when GLOBIGNORE is unset.

$ echo *
bin boot dev etc home lib lib64 lost+found mnt opt proc root run sbin srv sys tmp usr var
$ except=(etc lib lib64 tmp sbin)
$ GLOBIGNORE=$(IFS=:; printf "%s" "${except[*]}")
$ echo *
bin boot dev home lost+found mnt opt proc root run srv sys usr var

Of course, if you're creating the array, then you could directly set the GLOBIGNORE variable instead:

GLOBIGNORE=etc:lib:lib64:tmp:sbin

You can also take advantage of any patterns there might be in the list that fit bash wildcards:

GLOBIGNORE=etc:lib*:tmp:sbin
  • 1
    It looks like you can escape :, ?, *, [, ], (, )and backslash with backslash in GLOBIGNORE. So it looks like you should be able to construct a GLOBIGNORE for arbitrary array elements by doing the proper escaping before joining the elements. – Stéphane Chazelas Aug 10 '17 at 14:32
  • Beware that having GLOBIGNORE=.* means that ./* globs will fail to match and that dir/* will still include hidden file. That part of the manual is misleading. You don't want to have .* in GLOBIGNORE. Generally, GLOBIGNORE is quite useless in bash (compare with ksh's FIGNORE), but it looks like you've found one usage for it. – Stéphane Chazelas Aug 10 '17 at 14:37
  • @StéphaneChazelas pity ** doesn't work in GLOBIGNORE, it would have been interesting then. – muru Aug 10 '17 at 14:55
  • It's not needed. GLOBIGNORE is a pattern that is matched on the full glob expansion in the end, so */foo will exclude x/foo and x/y/z/foo just the same (it works like the ~ operator in zsh, you can get bash's GLOBIGNORE behaviour in zsh by appending ~$~GLOBIGNORE to your globs there). If you wanted to exclude x/foo only, you'd need *([^/])/foo (./x/foo would still be matched by ./**/foo though). – Stéphane Chazelas Aug 10 '17 at 15:00
  • @StéphaneChazelas it does? I tested GLOBIGNORE=*/.git* in a directory with a bunch of git repos in it, and for printf "%s\n" c*/* ./c*/* | grep git, I got ./cassandra/.git, ./certbot/.gitignore, etc. in the output, but not cassandra/.git or the like. – muru Aug 10 '17 at 15:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.