How can I check in bash whether a variable contains a valid UTF-8 string without any special control characters (such as newline or backspace or carriage return etc.)?
Provided the locale uses UTF-8 as its character encoding (check with
[[ $string =~ ^[^[:cntrl:]]*$ ]]
should work except that at least on GNU systems, it fails to reject strings that contain the UTF-8 encoding of characters with code points above 0x110000 (which are no longer valid in the current definition of UTF-8 encoding). That's down to the system's regexp library
bash uses to do pattern matching.
That's the same for GNU
expr " $string" : ' [^[:cntrl:]]*$' > /dev/null
Here, you may want to switch to
zsh where, in UTF-8 locales,
set -o extendedglob [[ $string = [' '-$'\ud7fff'$'\ue000'-$'\U10FFFF']# && $string != *[[:cntrl:]]* ]]
should work consistently regardless of the system (at least for the matching of valid characters; you might find variations between systems for which character is classified as
In zsh glob patterns, character ranges in multibyte character locales are based on the the wide character value of the character, which in UTF-8 is invariably the Unicode code point. Bytes that don't form part of valid characters are assigned wchar_t values in the 0xDC80..0xDCFF range in the second part of the 0xD800..0xDFFF UTF16 surrogate pairs which are not characters (a common approach sanctioned by Unicode to be able to handle arbitrary sequences of bytes).
Character range handling in bash glob patterns is mostly random, so can't be used here.
You may also want to consider what to do with non-characters, private use characters, characters that are not currently assigned in the version of Unicode your system is working with. The notion of control character is also quite fuzzy when it comes to Unicode. Are the U+202E RIGHT-TO-LEFT OVERRIDE character, the U+FEFF BOM character, the tag characters for instance control characters?