[Freeciv-Dev] Re: (PR#1824) ruleset data is in incompatible charsets

["Jason Short" <jdorje@xxxxxxxxxxxxxxxxxxxxx>]

<URL: http://rt.freeciv.org/Ticket/Display.html?id=1824 >

Raimar Falke wrote:
> <URL: http://rt.freeciv.org/Ticket/Display.html?id=1824 >
> 
> On Fri, May 07, 2004 at 11:32:30AM -0700, Jason Short wrote:
> 
>>So with these changes the flow chart becomes much simpler.  See
>>http://freeciv.org/~jdorje/iconv.png.
>>
>>Strings are stored in the server in unicode.  In the client they are
>>stored in the GUI encoding.
> 
> There is a big error and a problem in this.
> 
> The error is that GUI encoding will work with the functions in
> common/. We know from the past that this isn't the case. And it will
> also not be the case if the GUI encoding is latin1 (xaw) and the
> encoding used by common/ is utf8. The client and the server core needs
> the same encoding.

Why won't it work?  What happened in the past?

> The problem here is that the common client code can't convert into the
> GUI encoding since the function signatures have to change for
> different encodings. For ASCII and utf8 it is "char *" for utf16 and
> ucs2 it is however "short int *".

This isn't a problem so long as we assume we're always using 8-bit 
encodings.


The method you outline is possible.  However it will require many more 
changes, since all GUIs will have to be changed to convert to/from the 
internal encoding.  And I mean MANY changes.  And it will make the GUI 
code harder to maintain because of all the places where the conversions 
must be done.

If you plan to change all internal freeciv strings to be utf-16/ucs2 
then you're talking about many more changes again.  And here I mean MANY 
MANY changes.  UCS2 gives us no advantage over UTF-8, since the constant 
width doesn't do anything useful for us.  It also adds a hard dependency 
of iconv (as opposed to a soft dependency).

So we have to find the happy compromise between having the prettiest 
flowchart and having the prettiest code.

jason