For creating a C++ program that is source code level portable between Windows and Linux and handles internationalization well, there are IMHO three main encodings to consider:
- The encoding of the C++ source code.
- The encoding of external data.
- The encoding(s) of strings and literals.
For the C++ source code there is not really any alternative to UTF-8 with BOM, at least if standard input and wide string literals should work on the Windows platform. UTF-8 without BOM causes Microsoft's Visual C++ compiler to assume Windows ANSI encoding for the source code, which is nice for UTF-8 output via std::cout
, to the limited degree that that works (Windows console windows has lots of bugs here). However, then input via std::cin
does not work.
And for the external data UTF-8 seems to be the de facto standard.
However, what about the internal literals and strings? Here I had the impression that narrow strings encoded as UTF-8 was the common convention in Linux. But recently two different persons have claimed otherwise, one claiming that the common convention for internal strings in international applications in Linux is UTF-32, and the other just claiming that there is some unspecified difference between Unix and Linux in this area.
As one who fiddles a little, on a hobby basis, with a micro-library intended to abstract away the Windows/Linux differences in this area, I … have to ask concretely
- what is the common Linux convention for representing strings in a program?
I am pretty sure that there is a common convention that is so overwhelmingly common that this question has a Real Answer™.
An example showing e.g. how to Linux-conventionally reverse a string (which is complex to do directly with UTF-8, but which presumably is done by functions that are de facto standard in Linux?), would also be nice, i.e., as a question, what is a Linux-conventional version of this C++ program (the code as given works for Latin-1 as the C++ narrow text execution character set):
#include <iostream>
#include <algorithm>
#include <string>
using namespace std;
#define STATIC_ASSERT( cond ) static_assert( cond, #cond )
int main()
{
string line;
if( getline( cin, line ) )
{
static char const aSingleChar[] = "æ";
STATIC_ASSERT( sizeof( aSingleChar ) - 1 == 1 );
reverse( line.begin(), line.end() );
cout << line << endl;
}
}
std::string
has no notion of encoding; it should be understood as a "byte sequence". Reversing a string is not the same as reversing a run of text. As for source code and literals: C++ has a notion of "source encoding", which is opaque and unspecified, and any sane compiler should have a configuration option to specify the source encoding.-finput-charset
and-fexec-charset
, so you can always be explicit if you find the default insufficient.