[Freeciv-Dev] Re: (PR#1824) ruleset data is in incompatible charsets

["Vasco Alexandre da Silva Costa" <vasc@xxxxxxxxxxxxxx>]

<URL: http://rt.freeciv.org/Ticket/Display.html?id=1824 >

On Fri, 7 May 2004, Raimar Falke wrote:

> <URL: http://rt.freeciv.org/Ticket/Display.html?id=1824 >

> Can you explain the reason for ascii? I have the feeling that you have
> too much of the current model in your mind. If we do this we should do
> it right. So instead of adding two types (black+blue and black+red) we
> should only have one (blue in the server, red in the client). This
> will help the understanding a lot ("no field name of struct nation is
> ascii because it is read from the ruleset but field name of struct
> team is unicode because ..." better would be "all strings are foo at
> the server, bar on the wire and foobar at the client").

Same here. I think the server should use UTF-8, the wire UTF-8 and the
client whatever it likes. UTF-8 for GTK+ 2.2, UTF-16 for SDL, locale
encoding for the others. Easier this way.

The server could send and receive strings to the network as is, the
clients would use the current hooks in dataio to convert UTF-8 to the
encoding they use.

Since the client and server will need to use syscalls for several things,
like console I/O, file opening, etc, we need functions that convert:

for the client:
client encoding (UTF-8, UTF-16, locale encoding) <-> locale encoding.

for the server:
server encoding (UTF-8) <-> locale encoding


Then we just wrap all syscalls with this. For e.g.

char *filename, *name;

name = fc_from_locale(argv[i]);

fopen(fc_to_locale(filename), "r").

Data saved to or loaded from files is an exception. We actually want it
to be in UTF-8 to have portable savedgames and rulesets.

Amiga, Xaw, GTK+ 1.2 and Win32 will not even need any of this, since by
coincidence locale encoding == client encoding. Just common and the other
clients are problems.


We just need two functions, fc_to_locale and fc_from_locale for both
client and server. Then you have a function fc_locale_init that does this:

for the client:
fc_locale_init("UTF-16");       // for SDL.

for the server:
fc_locale_init("UTF-8");        // the server locale is UTF-8.


Do not despair. There are not *that* many syscalls with strings as
arguments that would need changing. :-)

---
Vasco Alexandre da Silva Costa @ Instituto Superior Tecnico, Lisboa