[Freeciv-Dev] (PR#10971) gtk client always crashes on bad string conversion

["Jason Short" <jdorje@xxxxxxxxxxxxxxxxxxxxx>]

<URL: http://rt.freeciv.org/Ticket/Display.html?id=10971 >

> [th@xxxxxxxxxxxx - Wed Nov 10 22:11:19 2004]:
> 
> On buying things civclient crashed every time with
> memory access violation
> 1: Invalid string conversion from UTF-8 to ISO-8859-15.
> 1: letzte Mitteilung 2mal wiederholt
> 1: letzte Mitteilung 2mal wiederholt (insgesamt 4 Wiederholungen)
> 1: letzte Mitteilung 4mal wiederholt (insgesamt 8 Wiederholungen)
> 
> drillich@mzpc>/opt/freeciv-2.0/bin/civclient --version
> Freeciv Version 2.0.0-beta3 (Beta Version) gui-gtk
> 
> Debian Linux 2.6.9-9-amd64-k8 #1 x86_64 GNU/Linux

What is the output of "echo $LANG"?

The root problem is that the gtk client uses the local encoding which
may not show all of the characters used in the rulesets.  The ruleset
names are not translated but may include non-ascii characters (currently
from latin1 or latin2).  This is also a problem for the win32 client. 
It's an even bigger problem for the XAW client since AFAICT it will
*only* display latin1 characters.

There are many possible solutions to this problem.

One solution is to just ignore it.  You (the user) can fix the problem
by changing your encoding to UTF-8.  If "echo $LANG" currently says
ru_RU.ISO-8859-15, change it to ru_RU.UTF-8.  This will work only for
the gtk client however.  You can also upgrade to using the gtk2 client
which will not have any of these problems (./configure
--enable-client=gtk2).

Another solution is to try to change the locale automatically within
freeciv.  However I think this would be very bug-prone.

One solution is to get rid of non-ascii names.  Names should probably be
translated - this puts a huge burden on the translators but would allow
names to work in languages that have entirely different alphabets (like
Russian or Japanese).  Currently not only is a Japanese player forced to
see European names in the latin alphabet, he's forced to see Japanese
names in the latin alphabet as well!

One solution is to use transliteration when converting the strings. 
This patch does that.  Transliteration will cause iconv to pick a
"matching" character for any character that it can't convert.  The
problem is transliteration support is inconsistent.  In GNU libiconv
(used by the win32 client) it is quite good and will convert accented
characters to the same character without the accent.  In glibc iconv
(which is what most linux users will have) it just converts all unknown
characters into question marks.  And I seem to remember sometimes in my
tests back when I first wrote the fciconv code it would completely crash.

jason

Index: utility/fciconv.c
===================================================================
RCS file: /home/freeciv/CVS/freeciv/utility/fciconv.c,v
retrieving revision 1.7.2.1
diff -u -r1.7.2.1 fciconv.c
--- utility/fciconv.c   15 Oct 2004 22:47:46 -0000      1.7.2.1
+++ utility/fciconv.c   11 Nov 2004 16:54:34 -0000
@@ -271,16 +271,24 @@
 #define CONV_FUNC_MALLOC(src, dst)                                          \
 char *src ## _to_ ## dst ## _string_malloc(const char *text)                \
 {                                                                           \
+  const char *encoding1 = (dst ## _encoding);                              \
+  char encoding[strlen(encoding1) + strlen("//TRANSLIT") + 1];             \
+                                                                           \
+  my_snprintf(encoding, sizeof(encoding), "%s//TRANSLIT", encoding1);      \
   return convert_string(text, (src ## _encoding),                          \
-                       (dst ## _encoding), NULL, 0);                       \
+                       (encoding), NULL, 0);                               \
 }
 
 #define CONV_FUNC_BUFFER(src, dst)                                          \
 char *src ## _to_ ## dst ## _string_buffer(const char *text,                \
                                           char *buf, size_t bufsz)         \
 {                                                                           \
+  const char *encoding1 = (dst ## _encoding);                              \
+  char encoding[strlen(encoding1) + strlen("//TRANSLIT") + 1];             \
+                                                                           \
+  my_snprintf(encoding, sizeof(encoding), "%s//TRANSLIT", encoding1);      \
   return convert_string(text, (src ## _encoding),                          \
-                        (dst ## _encoding), buf, bufsz);                    \
+                        encoding, buf, bufsz);                             \
 }
 
 #define CONV_FUNC_STATIC(src, dst)                                          \