[Freeciv-Dev] Re: (PR#1824) Re: Re: (PR#2743) Blank messages and bad packet strings

["Ben Webb via RT" <rt@xxxxxxxxxxxxxx>]

On Sun, Jan 12, 2003 at 07:14:46PM +0000, Vasco Alexandre Da Silva Costa wrote:
> UTF-8 makes sense for I/O in comms and for disk storage. Text output to
> console and the GUI for e.g. should be in the locale codeset.

Well, unless your GUI doesn't use the locale's codeset (e.g. GTK+2.0
always uses UTF-8 regardless, but I'm sure you know this).

> But is it not a pain to use internally? For e.g. you will need some string
> manipulation functions. You cannot rely on the standard C library for this
> so you will have to implement these yourself. How have you handled this in
> your projects?

UTF-8 strings are normal boring C strings; all of the normal C library
functions (e.g. strlen, strdup, strcpy, strcat, strstr) work just fine.
The only problems arise if you want to replace individual characters
within a string, as they may have different lengths in UTF-8. I never
need to actually do that in my projects though, although glib has lots
of nice functions for UTF-8 manipulation. Obviously there is a
difference between the number of characters and the number of bytes in a
UTF-8 string, but that is generally only an issue when dealing the UI.
Does the Freeciv server need to know the number of characters in a
string at any point? All of the registry/network/memory handling code
should be quite happy to deal in bytes.

(If you really want to subject yourself to my non-peer-reviewed code,
you can find it at http://dopewars.sf.net/. UTF-8 is sufficiently
straightforward to deal with that the same code will compile with either
GTK+1.x or GTK+2.x, using the locale's codeset internally in the first
instance and UTF-8 in the second. My Win32 API Unicode wrappers that I
mentioned are at src/gtkport/unicodewrap.[ch])

        Ben
-- 
ben@xxxxxxxxxxxxxxxxxxxxxx           http://bellatrix.pcl.ox.ac.uk/~ben/
"Heavier-than-air flying machines are impossible."
         - Lord Kelvin, president, Royal Society, 1895.