how to force shell script characters encoding from within the script

Question

I have a few shell scripts with UTF8-encoded characters inside and i want to be sure that they are decoded correctly regardless of the machine locale settings.

Is it possible to force the shell (bash or sh) to detect the correct script encoding? (something similar to the python or ruby encoding cookie)

The solution could be a custom shebang like:

#!/bin/bash --utf8

The solution should aim to better portability, so it is not necessary to stick with bash.

EDIT: maybe i've found a possible solution using a recursive script call:

# check if current locale is UTF8-based (otherwise this script may not work correctly)
locale | grep -q 'UTF-8'
if [ $? -ne 0 ]; then
    export LC_ALL=en_GB.UTF-8
    # recursive call this script with the modified environment
    $0 "$@"
    exit $?
fi

Answer 1

Bash stores strings as byte strings, and performs operations according to the current LC_CTYPE setting. So there is no need to restart bash: just set the LC_CTYPE or LC_ALL variable to your desired locale. Note that if you store a string in a variable or function, what matters is the encoding at the time the variable is expanded or the relevant command in the function is executed. Here's a script that demonstrates that:

#!/bin/bash
LC_CTYPE=en_US.utf8
v_utf8='é'
n_utf8=${#v_utf8}
f_utf8 () { tmp='é'; echo ${#tmp}; }
echo "UTF-8 in UTF-8: $n_utf8 $(f_utf8)"
LC_CTYPE=en_US
v_latin1='é'
n_latin1=${#v_latin1}
f_latin1 () { tmp='é'; echo ${#tmp}; }
echo "Latin 1 in Latin 1: $n_latin1 $(f_latin1)"
echo "UTF-8 in Latin 1: ${#v_utf8} $(f_utf8)"
LC_CTYPE=en_US.utf8
echo "Latin 1 in UTF-8: ${#v_latin1} $(f_latin1)"

Output:

UTF-8 in UTF-8: 1 1
Latin 1 in Latin 1: 2 2
UTF-8 in Latin 1: 2 2
Latin 1 in UTF-8: 1 1

As you can see, the length of the string is calculated according to the current value of LC_CTYPE, regardless of the value at the time of definition.