3

I'm trying to determine the root cause of why this find command is not working; it shouldn't match the file called this_should_not_match below:

$ > find . -type f -name "*[^ -~]*"
./__º╚t
./this_should_not_match
./__╞_u
./__¡VW
./__▀√Z
./__εè_
./__∙Σ_
./__Σ_9
./__Σhm
./__φY_

My shell is Bash 3.2

5

Ranges only work reliably and portably in the C locale. In other locales, you get some variation, but generally [x-y] gets you the characters (actually collating elements, it could even match sequences of characters) that sort after x and before y in some sort order which is often obscure and not always the same as sort would use.

In the C locale (see What does “LC_ALL=C” do?), characters are bytes and ranges are based on the code point of the characters (on byte values).

LC_ALL=C find . -type f -name "*[^ -~]*"

on an ASCII-based system (most of them; POSIX doesn't guarantee the C locale to use ASCII charset, but in practice, unless you're on some EBCDIC based special IBM mainframe OS (but then you'd know about it), you'll be using ASCII) would list regular files whose name contains bytes other than those between 32 and 126.

Also note that in a multi-byte character locale (like UTF-8 ones, the norm nowadays), the * may not even match all file names as on some systems, it will fail to match sequences of bytes that don't form valid characters.

  • The pattern is varied in shells. With LC_ALLL=C "$shell" -c 'case a in [!" "-~]) echo yes;; esac', only bash, ksh and yash say yes. – cuonglm Mar 8 '16 at 14:35
  • This works. Looks like $LC_ALL wasn't set to anything in my case – Zaid Mar 8 '16 at 14:40
  • @Zaid In which case it should default to C, shouldn't it? – Murphy Mar 8 '16 at 14:43
  • @Murphy I wouldn't know - I've never had to deal with locales before – Zaid Mar 8 '16 at 14:45
  • 1
    @cuonglm, zsh ranges are based on code-points. In UTF-8, bytes that don't form part of valid characters are mapped to would-be-code-points DC80 to DCFF (not valid characters), something to bear in mind when using a range that spans that like [$'\u1000-\U10000']. You'd want to use [$'\u1000-\ud7ff\ue000-\U10000'] to exclude the invalid [$'\ud800-\udfff'] range. You can also match them with byte ranges like [$'\x00-\xFF'] but note that it won't match on bytes that are otherwise part of multi-byte characters (you should only use byte-ranges in single-byte character locales). – Stéphane Chazelas Mar 8 '16 at 15:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.