On this (It is not intended to be a range, but an explicit list):
$ a='0123456789 ٠١٢٣٤٥٦٧٨٩ ۰۱۲۳۴۵۶۷۸۹ ߀߁߂߃߄߅߆߇߈߉ ०१२३४५६७८९'
$ echo "${a//[0123456789]}"
۰۱۲۳۴۵۶۷۸۹ ߀߁߂߃߄߅߆߇߈߉ ०१२३४५६७८९
Bash is incorrectly (IMO) removing the digits ٠١٢٣٤٥٦٧٨٩
(the second group).
The characters are all different (hand formatted):
$ for c in $(echo "$a" | grep -o .); do printf '\\U%04x ' "'$c"; done; echo
\U0030 \U0031 \U0032 \U0033 \U0034 \U0035 \U0036 \U0037 \U0038 \U0039
\U0660 \U0661 \U0662 \U0663 \U0664 \U0665 \U0666 \U0667 \U0668 \U0669
\U06f0 \U06f1 \U06f2 \U06f3 \U06f4 \U06f5 \U06f6 \U06f7 \U06f8 \U06f9
\U07c0 \U07c1 \U07c2 \U07c3 \U07c4 \U07c5 \U07c6 \U07c7 \U07c8 \U07c9
\U0966 \U0967 \U0968 \U0969 \U096a \U096b \U096c \U096d \U096e \U096f
Which correspond to:
123456789 # Hindu-Arabic Arabic numerals
٠١٢٣٤٥٦٧٨٩ # ARABIC-INDIC
۰۱۲۳۴۵۶۷۸۹ # EXTENDED ARABIC-INDIC/PERSIAN
߀߁߂߃߄߅߆߇߈߉ # NKO DIGIT
०१२३४५६७८९ # DEVANAGARI
To ensure there are no problems with pasting from this website, it is also possible to produce this Unicode content into the a
variable using the Unicode escapes:
a=$(echo -e '\u0030\u0031\u0032\u0033\u0034\u0035\u0036\u0037\u0038\u0039 \u0660\u0661\u0662\u0663\u0664\u0665\u0666\u0667\u0668\u0669 \u06f0\u06f1\u06f2\u06f3\u06f4\u06f5\u06f6\u06f7\u06f8\u06f9 \u07c0\u07c1\u07c2\u07c3\u07c4\u07c5\u07c6\u07c7\u07c8\u07c9 \u0966\u0967\u0968\u0969\u096a\u096b\u096c\u096d\u096e\u096f')
Or using the $'...'
strings which accept escapes directly:
a=$'\u0030\u0031\u0032\u0033\u0034\u0035\u0036\u0037\u0038\u0039 \u0660\u0661\u0662\u0663\u0664\u0665\u0666\u0667\u0668\u0669 \u06f0\u06f1\u06f2\u06f3\u06f4\u06f5\u06f6\u06f7\u06f8\u06f9 \u07c0\u07c1\u07c2\u07c3\u07c4\u07c5\u07c6\u07c7\u07c8\u07c9 \u0966\u0967\u0968\u0969\u096a\u096b\u096c\u096d\u096e\u096f'
Other shells do not work as bash (hand formatted):
$ for sh in zsh ksh lksh mksh bash; do $sh -c 'a="0123456789 ٠١٢٣٤٥٦٧٨٩ ۰۱۲۳۴۵۶۷۸۹ ߀߁߂߃߄߅߆߇߈߉ ०१२३४५६७८९"; echo "$0 : ${a//[0123456789]}" $sh'; done
zsh : ٠١٢٣٤٥٦٧٨٩ ۰۱۲۳۴۵۶۷۸۹ ߀߁߂߃߄߅߆߇߈߉ ०१२३४५६७८९
ksh : ٠١٢٣٤٥٦٧٨٩ ۰۱۲۳۴۵۶۷۸۹ ߀߁߂߃߄߅߆߇߈߉ ०१२३४५६७८९
lksh : ٠١٢٣٤٥٦٧٨٩ ۰۱۲۳۴۵۶۷۸۹ ߀߁߂߃߄߅߆߇߈߉ ०१२३४५६७८९
mksh : ٠١٢٣٤٥٦٧٨٩ ۰۱۲۳۴۵۶۷۸۹ ߀߁߂߃߄߅߆߇߈߉ ०१२३४५६७८९
bash : ۰۱۲۳۴۵۶۷۸۹ ߀߁߂߃߄߅߆߇߈߉ ०१२३४५६७८९
The bash sort order is:
$ mkdir test1; cd test1; IFS=$' \t\n'
$ touch $(echo "$a" | grep -o .)
$ printf '%s' *; echo
߃߇߆߁߂߅߉߄߀߈0٠०۰1١१۱٢2२۲3٣३۳٤4४۴٥5५۵٦6६۶7٧७۷8٨८۸٩9९۹
$ locale
LANG=en_US.utf8
LANGUAGE=
LC_CTYPE="en_US.utf8"
LC_NUMERIC="en_US.utf8"
LC_TIME="en_US.utf8"
LC_COLLATE="en_US.utf8"
LC_MONETARY="en_US.utf8"
LC_MESSAGES="en_US.utf8"
LC_PAPER="en_US.utf8"
LC_NAME="en_US.utf8"
LC_ADDRESS="en_US.utf8"
LC_TELEPHONE="en_US.utf8"
LC_MEASUREMENT="en_US.utf8"
LC_IDENTIFICATION="en_US.utf8"
LC_ALL=
It doesn't seem to be applying the sort order to remove characters.
It shouldn't anyway (IMO) as the characters are being explicitly listed.
So: Why?
Using bash 4.4.12 here. But it fails also with 3.0, 3.2, 4.0, 4.1, 4.4.23, 5.0 but not with 2.0.1 nor 2.0.5. It seems that a change in 3.0 caused the issue.
LANG
setting of your environment?a='0123456789 ٠١٢٣٤٥٦٧٨٩ ۰۱۲۳۴۵۶۷۸۹ ߀߁߂߃߄߅߆߇߈߉ ०१२३४५६७८९'; b="${a//[0123456789]}"; echo "${#a} ${#b}"
outputs54 44
which is what you would expect. I do see your issue on Ubuntu 17.10, 4.4.12(1)-release and en_US.UTF-8 where the output of my command is54 34
. I spotted a report here for C.UTF-8 but I don't know if the underlying issue is relevant.echo -e "$(printf '\\U%s' {{3{0..9},20,66{0..9},20,6f{0..9},20,7c{0..9}},20,96{{6..9},{a..f}}} )"
?