[Freeciv-Dev] Re: (PR#1824) ruleset data is in incompatible charsets

["Vasco Alexandre da Silva Costa" <vasc@xxxxxxxxxxxxxx>]

<URL: http://rt.freeciv.org/Ticket/Display.html?id=1824 >

On Sat, 8 May 2004, Raimar Falke wrote:

FWIW I agree with:
<http://www.freeciv.org/~rfalke/iconv-rf.png>

If "Network" is in UTF-8 and "Unicode" is in UTF-8 as well. Why? Because it
makes no sense to use UCS-2. It gives nothing. You need to change the
code all over the map and make your own complete C library for string
handling functions. If we use UTF-8, many functions such as strlen(),
will still work.

> For utf8 number of chars isn't number of bytes. There were problems
> with some buffers. Also is_sane_name uses is_iso_latin1.

Is that the problem? There is a maximum number of bytes a character can
take in UTF-8:
<http://en.wikipedia.org/wiki/UTF-8>

4 bytes. Just increase the size of all string buffers * 4. So, it is twice
as much as UCS-2. Big deal. UCS-2 is twice as big as ISO-8859-1. :-)

> You agree that it is the cleaner approach?!
>
> For gui-sdl and gui-fs these changes aren't this much of a problem
> since all string data already goes through only a few functions.

Using UTF-8, everywhere you can, makes more sense. No need for network
translations either.

> This decision is independent of the above. And I agree with you that
> utf-16 would require a lot of changes.
>
> > So we have to find the happy compromise between having the prettiest
> > flowchart and having the prettiest code.
>
> _If_ we choose utf8 for the blue encoding the gtk2 client doesn't have
> much problems. The conversion from blue to red is a nop. xaw and gtk1
> would have to be changed.

Precisely. In your scheme, with blue, brown and red all in UTF-8
for the current civserver + gui-gtk-2.0 client there are zero conversions.

You only need conversions for syscalls.

---
Vasco Alexandre da Silva Costa @ Instituto Superior Tecnico, Lisboa