Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

Qt applications are deleting non-latin characters from ISO-8859 encoded files on my Gentoo system. Actually I'm trying to merge two German files with KDiff3 and P4Merge (making "Whlen" out of "Wählen"). Both tools don't display the Umlauts and when the file is saved, they also disappear in the file. The Dejavu Monospace font is used, Courier New shows the same behavior.

If UTF-8 encoded files are presented to the tools, all non-latins are handled correctly.

GTK Meld (and all other GTK apps) handles the chars (ISO-8859-1 and UTF-8) quite well. I believe its my Locale configuration, but can't discern what's amiss...

Any ideas?

Configs:

 $ locale -a
 C
 POSIX
 de_DE
 de_DE.iso88591
 de_DE.iso885915@euro
 de_DE.utf8
 de_DE@euro
 deutsch
 en_US
 en_US.iso88591
 en_US.utf8
 german

 $ locale
 LANG=en_US.UTF-8
 LC_CTYPE="en_US.UTF-8"
 LC_NUMERIC="en_US.UTF-8"
 LC_TIME="en_US.UTF-8"
 LC_COLLATE=C
 LC_MONETARY="en_US.UTF-8"
 LC_MESSAGES="en_US.UTF-8"
 LC_PAPER="en_US.UTF-8"
 LC_NAME="en_US.UTF-8"
 LC_ADDRESS="en_US.UTF-8"
 LC_TELEPHONE="en_US.UTF-8"
 LC_MEASUREMENT="en_US.UTF-8"
 LC_IDENTIFICATION="en_US.UTF-8"
 LC_ALL=

Qt use flags for x11-libs/qt-core-4.8.2:4:

exceptions glib iconv qt3support ssl (-aqua) -c++0x -debug -icu -pch
(-optimized-qmake%) (-qpa%)
share|improve this question
1  
Is switching the locale to match your input files not acceptable? –  Gilles Jan 11 '13 at 23:45
add comment

1 Answer

If UTF-8 encoded files are presented to the tools, all non-latins are handled correctly.

Then I suggest you recode the files to UTF-8, merge them, then recode back if neccessary.

GTK Meld (and all other GTK apps) handles the chars (ISO-8859-1 and UTF-8) quite well.

I guess you might call this a feature, not a bug: when the application sees some byte sequences it cannot interpret as characters in the current locale, it has several options. It might start guessing, which apparently is what GTK does. Or it might consider the corresponding bytes to be errors and drop them from the file. Or it might inform the user about these issues, which would be even nicer, but apparently KDE doesn't go this far.

I believe its my Locale configuration, but can't discern what's amiss...

Your locale indicates that text files are expected to be encoded as UTF-8. Your files however are not. So there is a discrepancy. You can fix that by either recoding the files, or by starting the process in question with a different locale. The LC_CTYPE facet should be the relevant one, but start with LC_ALL first just to be sure. When invoking the command from the command line, simply prepend the correct setting to the command, i.e.

LC_CTYPE=en_US.iso88591 kdiff3 foo bar
share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.