[Freeciv-Dev] (PR#9976) changing data encoding to utf-8

["Jason Short" <jdorje@xxxxxxxxxxxxxxxxxxxxx>]

<URL: http://rt.freeciv.org/Ticket/Display.html?id=9976 >

(I thought I sent this ticket earlier, but I can find no sign of it.  I 
guess I must have lost it somewhere.)

As discussed in PR#1824, we now need to change the data encoding to 
UTF-8.  Until this is done it is not possible for the server to use any 
non-latin1 characters.

It's a simple process.  We just apply the attached patch and run the 
attached conversion script in data/nation/.  This could use a new 
capability, but it probably doesn't matter too much.

I looked for documentation that needed changing but couldn't find any. 
Something should be added to doc/README.rulesets (this will also need 
updating after the effects patch).  From now on all rulesets must be in 
UTF-8.

This should be committed soon.

jason

Index: utility/fciconv.h
===================================================================
RCS file: /home/freeciv/CVS/freeciv/utility/fciconv.h,v
retrieving revision 1.2
diff -u -r1.2 fciconv.h
--- utility/fciconv.h   6 Sep 2004 17:13:07 -0000       1.2
+++ utility/fciconv.h   6 Sep 2004 17:25:32 -0000
@@ -15,7 +15,7 @@
 
 #include "shared.h"
 
-#define DEFAULT_DATA_ENCODING "ISO-8859-1"
+#define DEFAULT_DATA_ENCODING "UTF-8"
 
 void init_character_encodings(char *internal_encoding);
 

#!/usr/bin/perl

foreach (@ARGV) {
  print "Converting $_.\n";

  system("iconv -f latin1 -t UTF-8 $_ > $_.tmp");
  system("mv $_.tmp $_");
}