[Freeciv-Dev] Re: (PR#9804) Unicode in registry code

["Jason Short" <jdorje@xxxxxxxxxxxxxxxxxxxxx>]

<URL: http://rt.freeciv.org/Ticket/Display.html?id=9804 >

Mateusz Stefek wrote:
> <URL: http://rt.freeciv.org/Ticket/Display.html?id=9804 >
> 
> Here's my proposition about character encoding in registry code.
> 
> 1) Section and entry names will be saved in pure ASCII

I'm not sure what would happen if you tried to use UTF-8 here.  It's 
possible it would work.  But since the code is all ascii it might lead 
to problems.

> 2) String values will be saved in UTF-8
> 3) secfile_insert_str* will accept only UTF-8
>    This is not a big problem. Only the code which saves client options
> need to convert CMA presets' names into UTF-8
> 4) secfile_lookup_str* will return UTF-8
> 5) savegame compatibility
>    There will be function called secfile_set_fallback_encoding(file,
> encoding). If the string which is read is not valid utf8, it will be
> converted from that encoding.

Easier would be to use secfile_set_encoding().  This encoding is used 
for writing and reading.  The registry code can accept strings in the 
program's internal encoding and read/write them into the registry encoding.

Of course the registry encoding will be UTF-8.  Except when loading old 
savegames where it may be latin1 or something else.

> 6) utility functions needed:
> - utf8_validate

I don't think this is necessary.

> - is_ascii

This might be good for sanity checking.  Note the is_ascii() we have 
isn't designed to be an exact check for ascii validity but is more of an 
is_sane_ascii_name() function.

> - convert_string (it's in Jason's patch)

Easy.

jason