Padding unicode strings with bash's printf

Question

I tried padding a Unicode string with bash's printf, and saw that, while

printf "%2s" a

yields the expected " a",

a Unicode variant

printf "%2s" ä  

yields a surprisingly unpadded "ä". (zsh gives the expected result.)

What causes this; and how am I supposed to pad Unicode strings in bash?

Answer 1

The character ä is encoded with 2 bytes in UTF-8, so Printf takes it for 2-padded.

Wc can count the characters (-m) and bytes (-c) of a string. The number to give Printf is then [intended pad]+[bytes]-[chars]. So I have assembled this pad.sh script,

#!/bin/sh
bytes=$(printf '%s' "$2" | wc -c)
chars=$(printf '%s' "$2" | wc -m)
n=$(($1+bytes-chars))
printf "%${n}s" "$2"

In the example execution below, I have artifficially added newlines after each output for clarity's sake.

$ sh pad.sh 10 abcdef
    abcdef
$ sh pad.sh 10 äéßôçÈ
    äéßôçÈ

Answer 2

how am I supposed to pad Unicode strings in bash?

That's way beyond the capabilities of bash. If you're limiting "Unicode strings" to ascii++ (no double width characters, no bidi, no non-spacing marks, no etc), you may jury-rig something like:

% pad(){ printf '%*s%s\n' "$(($1-${#2}))" "" "$2"; }
% pad 2 €
 €

Answer 3

bash behaves correctly and the C program

#include <stdio.h>
main()
{
        char foo[] = "ä";

        printf("%2s\n", foo);
}

behaves the same.

This is because %s refers to a byte oriented string and 'ä' in UTF-8 results in 2 bytes.

As far as I could test, none of the other shells behaves incorrect.

The result you expect could be seen with something like:

printf '%2S\n' ä

but this is not supported by any of the shells I tested.