[Freeciv-Dev] Internationalization

[David Pfitzner <dwp@xxxxxxxxxxxxxx>]

Hi all,

With increased interest recently to internationalize Freeciv
(that is, add multi-language support), here are a few thoughts 
on some difficulties which may be involved.

First a disclaimer: I don't think we need to solve all of
these problems all at once.  It should be fine to add initial 
limited gettext support, with possibly only a few "easy" 
strings initially marked for translation, and then work on 
improvements from there.

People interested in doing this should probably try to 
coordinate.  People who I know have worked on this or 
indicated interest, now or previously, are in the CC.
(But there may be others.)

Ok, some problems:

1. Client-server issues.  It would be good to be able to
have clients and servers localized to different languages,
and still have players see all messages in their own language, 
including messages sent by the server.  The main issue here 
is whether the translation is done by the client or by the
server (where the client would tell the server what language 
to send).

A possible problem with the server doing the translation is 
that I think with gettext it may be inefficient to change 
which language is being used.  So if the server sends out 
lots of messages (eg at end-of-turn) and has to keep changing 
its language bindings, it may be too slow.  

A problem with the client doing the translation is that
by the time the message arrives at the client, it will
generally be hard to translate because the server has already
composed the message from several components, some of which
are translatable (eg, a printf format string, a tech name) 
and some of which should not be translated (eg, player names, 
city names, numbers).

(Actually, this would imply some re-structuring even if the
server did the translation when sending to clients, since
currently the server will typically compose a message from
components before knowing which client(s) the message will
be sent to.)

A possible solution is for the server to not send a fully 
built string, but to send a format string (probably not printf,
but maybe similar) and other strings to plug into the format, 
where the client then translates the format string, and selected 
of the other strings, I guess based on the format specifier,
before plugging in the extra strings.

Or maybe the server sends a single string, but with markup
to tell the client which parts to individually translate,
and which parts not to, and then the client removes the
markup before displaying.


2. Data files.  Currently unit names etc are stored in ruleset
files, which are read in by the server and then the information
is sent to clients.  For the client to have appropriate translations,
I think the translations need to be inside the ruleset files.
Otherwise the client may use completely inappropriate names if 
the server used a customized or changed ruleset file.

So probably, the ruleset files should have a section giving
lists of languages, and then a list of translated names for 
each language.  The server would read in all the translations, 
and send to a client the default-language names and the names 
in the client-prefered language, so the client can do translations.


3. Helptext.  There are really two types of helptext: that for 
units etc, and general text.  For the former, its currently
important the that building etc names in the helpdata.txt match 
the names used by the ruleset (or else one doesn't get specific 
helptext).  Really, all such information should be moved into
the ruleset files anyway, in which case it could be handled like
point 2, with translations also in the ruleset files. 

For generic helptext, each entry in the helptext could have multiple 
sections for different languages; or there could be multiple separate
helpdata_*.txt files, one per language; or multiple files where the 
default is read in first and then a language-specific version is 
read-in after and overrides some entries.   An issue here would be 
whether one of these methods makes it easier to keep translations
up-to-date, but I'm not sure which...


4. Sentence generation, grammatical contructions.  In some places 
freeciv automatically generates some sentence elements, such as
using "a" or "an" before a noun, or adding a location qualifier
to a message.  I guess this will cause some problems.


There may be other issues (eg, sorting "alphabetically"?),
but the above should do for starters :-)

Regards,
-- David