[Freeciv-Dev] Re: (PR#1824) Re: Re: (PR#2743) Blank messages and bad packet strings

["Jason Short via RT" <rt@xxxxxxxxxxxxxx>]

Raimar Falke via RT wrote:
> On Thu, Jan 09, 2003 at 11:17:02PM -0800, Jason Short via RT wrote:
> 
>>Raimar Falke wrote:
>>
>>
>>>I disagree if this means that the city name "K?ln" won't be longer
>>>allowed. Unicode with one of its encodings would be best?! The only
>>>problem is IMHO that this makes another dependency?!
>>
>>I agree the accented characters are nice.  But if you allow latin1 
>>characters why not latin2 characters?  Why not allow Japanese 
>>characters?  Where do you draw the line?
>>
>>But, I agree with you.  We should allow the ruleset to specify its 
>>encoding, and have freeciv convert from that encoding to the local 
>>encoding when loading it.  Where to draw the line is a data issue, and 
>>it's always possible for modpack authors to set their own standards.
>>
>>A good start would be to convert from latin1 into the local encoding 
>>when loading the ruleset.  That fixes PR#752/1824/2445/2743.  (I assume 
>>iconv can do something intelligent when converting an accented character 
>>into an ascii encoding.)
>>
>>For anyone interested in this issue, if you haven't been following it 
>>from the start you may want to re-read PR#1824 from the beginning. 
>>There's a lot of talk and no real solutions...
> 
> 
> I ask another time: Is unicode the best solution? Why don't we use it?

Unicode isn't "the solution".  "The solution" is converting from the 
charset the strings are in into the local charset.

Putting the charsets into unicode may be part of the solution.  But 
without the other part, this will keep the same problem but reverse who 
sees it (i.e., everyone who currently sees accented characters will see 
garbage, and most people who currently see garbage will see accented 
characters).

The system is further complicated because client and server may have 
different charsets.  Here the solution may be to put all 
charset-dependent strings into unicode.

jason