[Freeciv-Dev] (PR#9804) Unicode in registry code

["Jason Short" <jdorje@xxxxxxxxxxxxxxxxxxxxx>]

<URL: http://rt.freeciv.org/Ticket/Display.html?id=9804 >

> [mstefek - Wed Aug 25 08:03:51 2004]:
> 
> > [jdorje - Wed Aug 25 07:27:08 2004]:
> > 
> > Mateusz Stefek wrote:
> [..]
> > > 5) savegame compatibility
> > >    There will be function called secfile_set_fallback_encoding(file,
> > > encoding). If the string which is read is not valid utf8, it will be
> > > converted from that encoding.
> > 
> > Easier would be to use secfile_set_encoding().  This encoding is used 
> > for writing and reading.  The registry code can accept strings in the 
> > program's internal encoding and read/write them into the registry
> encoding.
> 
> It means that the registry code will convert or just check the encoding
> each time a string will be inserted. I'm not sure how it will affect
> game saving, but IMO it's not necessary. There is only one place when
> the conversion is needed: client option file.

But the alternative is that the calling code must do this conversion. 
Of course if the internal charset is UTF-8 this is not needed.  But IMO
it's cleaner to have the saving code take strings that are in the
internal charset (declared inside a variable in fciconv.h) and conver
them manually.  The conversion is fast.

> We really don't know in which encoding are old savegames saved.
> We can assume it's UTF-8 and if the string isn't valid, we treat it as
> latin1. It is really rare situation when the correct latin, but not pure
> ascii string is also correct in UTF8. This can only happen when two
> special characters are next to each other. (It quite rare in Polish and
> it looks like it is almost impossible in German)

Good point.  But is this rare case worth polluting the code?  It is easy
to give instructions on how to convert such savegames.  Just run

  iconv -f latin2 -t utf-8 mstefek.sav

jason