Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.
$ echo ABC | awk '$0 ~ /^[a-b]/'
ABC
$ echo ABC | awk '$0 ~ /^[a-a]/'
$ echo ABC | awk '$0 ~ /^a/'
$ 

You see. /[a-b]/ captures A, but /[a-a]/ or /a/ doesn't. Why?

share|improve this question
1  
See Does (should) LC_COLLATE affect character ranges? for more (unresolved) info on this topic. –  Caleb Aug 24 '11 at 21:07
    
This appears to be more than just a simple(?) LC_COLLATE issue, because using some non-C values for LC_COLLATE produces different results, depending on which utility is used. eg. 'sed' and 'grep' give different results to 'awk' when using LC_COLLATE=en_AU.UTF-8 or en_US.UTF-8 ... sed and grep manage to resolve the case issue, and only lower-case is printed (using the same values as above) –  Peter.O Aug 25 '11 at 9:38

1 Answer 1

up vote 6 down vote accepted

It is a "locale" problem, I think.

In my locale, it_IT, the following snippet

if [[ a < A ]]; then
  echo "a < A"
elif [[ a > A ]]; then
  echo "a > A"
else
  echo "a = A"
fi

if [[ b < A ]]; then
  echo "b < A"
elif [[ b > A ]]; then
  echo "b > A"
else
  echo "b = A"
fi

shows

a < A
b > A

so that A is (surprisingly) between a and b, so in the range.

Try executing

echo ABC | LC_COLLATE=C awk '$0 ~ /^[a-b]/'

Edit

the following command shows the collating order in your locale:

    echo $(LC_COLLATE=C printf '%s\n' {A..z} | sort)

the output on my machine is

    ` ^ _ [ ] a A b B c C d D e E f F g G h H i I j J k K l L m M n N o O p P q Q r R s S t T u U v V w W x X y Y z Z

(cannot understand from bash's manual page if sequence expressions are expanded in locale collating order or not; it seems not).

share|improve this answer
    
+1, but you just need LC_COLLATE, not LC_ALL, for this particular case. –  mattdm Aug 24 '11 at 15:40
    
@mattdm: you're right, i'm lazy –  enzotib Aug 24 '11 at 15:52
    
@enzotib: I am puzzled... This idea seems to mean that every time I want to set a range /[a-x]/, I must use LC_COLLATE....What on earth has a collating sequence got to do identifying what is Upper-case vs Lower case? ... I can't see how a Collating sequence dictates what is upper case and what isn't... I keep grappling with these locale issues, and am slowly making headway, but this one has me stumped. –  Peter.O Aug 24 '11 at 18:14
    
@fred: frequently, when using sort, join or the like, I start my scripts with export LC_COLLATE=C. Now I have to start this way also scripts using awk :) –  enzotib Aug 24 '11 at 18:44
2  
The sequence eval order doesn't matter in this case since you sort after the sequence is generated. However, your example would work more accurately with LC_COLLATE next to sort: "echo $(printf '%s\n' {A..z} | LC_COLLATE='C' sort)" ... which would contrast correctly with the default case "echo $(printf '%s\n' {A..z} | LC_COLLATE='' sort)". The original syntax above never actually applies the locally modified LC_COLLATE to the sort command [of course, all bets are off if LC_ALL was set somewhere...] –  MartyMacGyver Jun 1 '12 at 19:22

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.