[Freeciv-Dev] Re: (PR#7317) Add Unicode support to Freeciv

["Raimar Falke" <i-freeciv-lists@xxxxxxxxxxxxx>]

<URL: http://rt.freeciv.org/Ticket/Display.html?id=7317 >

On Mon, Jan 26, 2004 at 12:44:44AM -0800, Per I. Mathisen wrote:
> 
> <URL: http://rt.freeciv.org/Ticket/Display.html?id=7317 >
> 
> On Sun, 25 Jan 2004, Raimar Falke wrote:
> >  - ucs2 and ucs4 are fixed length encodings
> 
> What are these? I have never heard anything about them before.

UCS2 is the predecessor of UTF16. UTF16 was necessary after the
unicode consortium realized that they need more then 2^16 chars.

UCS4 is the same as UTF32 for practical purposes. Theoretically UTF32
is limited to 2^21 while UCS4 is defined till 2^32.

> > I now argue that the "variable length" is the issue which causes some
> > of the issues in the past and also will cause us problems in the
> > future. In the network code and in the server we want to ensure that
> > the received strings have a sane length. For this we static buffers
> > and functions like strncpy. Here utf8 shows it's problems. How do you
> > dimension these buffers? You would have to take the conservative
> > approach of the worst-case (multiple their size by 4). We have also
> > seen the problems of truncated utf8 strings.
> 
> What about utf-16 and utf-32? 

> utf-32 is fixed-length, afaik.

Correct.

> If we only use the first 16 bits worth of characters, utf-16 should
> be fixed-length too.

Than we have UCS2.

        Raimar

-- 
 email: rf13@xxxxxxxxxxxxxxxxx
 "Python 2.0 beta 1 is now available [...]. There is a long list of new 
  features since Python 1.6, released earlier today. We don't plan on 
  any new releases in the next 24 hours."
    -- Jeremy Hylton at Slashdot