[Freeciv-Dev] Re: (PR#2559) SDL_ttf font problem

["Davide Pagnin via RT" <rt@xxxxxxxxxxxxxx>]

I never thought that this issue will continue to persist for so long
time and with a continue spreading of new problems!!!

But ...

>  Uint16 *convert_to_utf16(const char *pString)
>  {
>    /* Start Parametrs */
> -  const char *pTocode = "UTF-16";
> -  const char *pFromcode = LOCAL_CODE;
> +#if SDL_BYTEORDER == SDL_LIL_ENDIAN
> +  const char *pTocode = "UTF-16LE";
> +#else
> +  const char *pTocode = "UTF-16BE";
> +#endif
> +#ifdef HAVE_LANGINFO_CODESET
> +  const char *pFromcode = nl_langinfo(CODESET);
> +#else
> +  const char *pFromcode = "";
> +#endif

I found that nl_langinfo use on my Tru64 Unix alpha system do causes
core. Discovering why has taken over 1 hour of my time...
And, even if I've found the cause, I'm not sure what do to as the
cure...

starting from the beginning:

nl_langinfo is implemented in the libc library, that is the native C
library of the Tru64 Unix OS.
It works as normally supposed and have not any strange behavior.

a call to nl_langinfo(CODESET) on my alpha system, return as result:
"ISO8859-1"
That is the correct answer.

what's going on then?

The native libiconv.so of the system, isn't good (or at least, isn't
enough good to do what is supposed to do for freeciv and gui-sdl) than
I've had the necessity to install GNU libiconv, and I've successfully
installed the latest version, 1.8.

After installing libiconv 1.8, I get this output from the GNU iconv
program:

iconv -l
...
ISO-8859-1
...

To explain it simple, there is no ISO8859-1 defined in the available
list of charset.

This means that issuing something like:

iconv_open("UTF-16LE","ISO8859-1")

On my system, will be equal to (iconv_t)-1 (conversion not available!)

The libiconv packager has made a different choice (than GNU libc iconv
packager) to solve the problems with charset aliases.
Within the distribution of GNU libiconv, there is another library, that
is called libcharset, and which help into trasforming into canonical
name the charset aliases typical of a system and not of another.

the recipe is:
include <libcharset.h>

const char * localcode = locale_charset();

Which, on my system, gives: ISO-8859-1
(Which is the correct answer and moreover an answer that will please
iconv_open)

Here we have many option to solve the portability issues and to support,
out the box, tha vast majority of the systems.

I do think that requiring GNU libiconv instead of using GNU libc iconv,
isn't a good idea, then we have to cope with the different way this two
GNU products works.

For GNU libc iconv, the "right" way is:
#include <langinfo.h>
const char * locale_code = nl_langinfo(CODESET);
(

For GNU libiconv (1.7 or above, I think), the "right" way is:
#include <libcharset.h>
const char * locale_code = locale_charset();

I do suggest to do a configure check for libcharset.h and
locale_charset, after that iconv has been found.
If this research is positive, good chances are that we are using GNU
libiconv and that locale_charset() will do the correct job.
If this fails, will failover to the GNU_libc model.

More, I do think that the configure check for nl_langinfo is there in
the event that we use intl/* stuff, perhaps we can learn something from
that code, also.


> @@ -210,8 +214,16 @@
>  char *convert_to_chars(const Uint16 * pUniString)
>  {
>    /* Start Parametrs */
> -  const char *pTocode = LOCAL_CODE;
> -  const char *pFromcode = "UTF-16";
> +#if SDL_BYTEORDER == SDL_LIL_ENDIAN
> +  const char *pTocode = "UTF-16LE";
> +#else
> +  const char *pTocode = "UTF-16BE";
> +#endif
> +#ifdef HAVE_LANGINFO_CODESET
> +  const char *pFromcode = nl_langinfo(CODESET);
> +#else
> +  const char *pFromcode = "";
> +#endif

Ok, here above, Tocode and Fromcode are swapped, as you can easily see
by looking to the old lines.
i.e pTocode was LOCAL_CODE and now is UTF-16XX.

And, for the moment, here we are.

        Ciao, Davide