[Freeciv-Dev] Configurable units etc: Rulesets patch

[David Pfitzner <dwp@xxxxxxxxxxxxxx>]

An updated version of my "rulesets" patch to allow user-configurable 
units etc without recompiling is now available at

  http://msowww.anu.edu.au/~dwp/freeciv/patches/

(files rulesets5.diff.gz etc)

It is patch vs current (~ Nov 7) cvs freeciv.  I won't post it here 
because its rather large (51 kB gzipped).  Currently included rulesets 
are "default" (standard) freeciv, and "civ1".

Since previous releases of this patch (~ August) it has had various 
improvements, structural revisions, keeping up with cvs etc.  I consider 
that its now in a state that (preferrably after a bit more testing) it 
could be included into CVS.

As I said in an email quoted by Chris Weber:

> > If 1.7.2 is going to come out soon then I'm thinking to include
> > my patch soon _after_ 1.7.2 (rather than introduce lots of new
> > bug opportunities for the release).

But if enough people want it I don't object to it being included
before 1.7.2   After watching the identical development of all-AI games 
with and without this patch, with the same random seeds and the default
rulesets, I'm reasonably confident that at least with the default
rulesets there aren't too many new bugs despite the many source changes.

Following is the README.rulesets file.

-- David


----------------------------------------------------------------------
Freeciv Rulesets, by David Pfitzner, dwp@xxxxxxxxxxxxxx
----------------------------------------------------------------------

Quickstart:
-----------
 Rulesets allow modifiable sets of data for units, advances, 
 improvements, and wonders, without requiring recompilation, in a way
 which is consistent across a network and through savegames.  (In the
 future there could be other categories of rulesets, eg terrain, 
 governments?)

- To play freeciv normally: don't do anything special; the new
  features all have defaults which give the standard freeciv 
  behaviour.

- To play a game with rules more like Civ1, start the server with:
       ./ser -r civ1.serv
  (and any other command-line arguments you normally use).  
  Start the client normally, using a client compiled from a source 
  distribution which includes rulesets.  A non-rulesets-aware client 
  will be rejected if it tries to connect to a rulesets-aware server,
  and vice-versa.  
  Note that the freeciv AI might not play as well with rules other
  than standard-freeciv.  (See notes below.)

The rest of this file contains:

- More detailed information on creating and using custom/mixed 
  rulesets.

- Information on implementation, and notes for further development.

----------------------------------------------------------------------
Using and modifying rulesets:
-----------------------------

Rulesets are specified using new server options.  The command above
of "./ser -r civ1.serv" just reads a file which sets these options 
(as well as a few of the standard server options).  The new server 
options are:

        techs, units, buildings.

They are special server options in that they take string values, but 
otherwise the behave like normal server options.  Eg, try the server 
command "help techs".

For each of these options, the value of the option specifies the name
of a file in the freeciv data directory, where the full filename is:

        (option_value)_(option_name).ruleset

Eg, the commands:
   
        set techs default 
        set units civ1
        set buildings custom

would specify the files:

        data/default_techs.ruleset
        data/civ1_units.ruleset
        data/custom_buildings.ruleset

(This is just an example, and may not give very sensible rules; the
file data/custom_buildings.ruleset doesn't exist unless you create 
it.)

The ruleset files in the data directory are user-editable, so you can
modify them to create modified or custom rulesets (without having to
recompile freeciv).  It is suggested that you _don't_ edit the 
existing files with prefixes "default" and "civ1", but rather copy 
them and edit the copies.  This is so that its clear when you are 
using modified rules and not the standard ones.

The format used in the ruleset files should be fairly 
self-explanatory.  A few points:

- The files are not all independent, since eg, units depend on
  advances specified in the techs file.

- Wonders and city improvements have a new field, "variant".  This 
  allows limited changes to the effects of specific wonders and 
  improvements, where such changes have been implemented.  The help 
  system (and also the "TODO Variants" section below) specify the 
  variant effects which have been implemented so far.

- Units have a new field, "roles", which is like "flags", but 
  determines which units are used in various circumstances of the
  game (rather than intrinsic properties of the unit).
  See comments in common/unit.h

- The units file contains 4 tables.  These should all contain the
  same unit names in the same order.

- The [unit_adjustments] section of the units file deserves some
  explanation.  It contains the entries:

        max_hitpoints, max_firepower, firepower_factor.

  The first two entries (unless zero) override the hitpoints and 
  firepower entries in the units.u2 data-table, as a convenience.  
  The value of firepower_factor is used when combat is resolved:
  All combat firepower values are multiplied by firepower_factor,
  which means that the effective hitpoints of any unit is really 
      (hitpoints/firepower_factor).
  Actually, it is these effective hitpoints which are report to the
  client for all hitpoint values.
  (This complicated setup is so that the AI calculations don't 
  blow up when using Civ1 rulesets.)

Note that some properties of units, advances, etc are still compiled
in properties (eg, see the TODO section below), so when making custom
rulesets you should aim to make changes as limited as possible, to 
have the best chance of the changes working properly.

----------------------------------------------------------------------
The civstyle option:
--------------------

The server option civstyle still exists, but, as before, it doesn't
actually do anything.  Rulesets allows civ1 style rules (to some 
approximation) with more flexibility that a single server option.
Potentially the civstyle option could still be put to some use in 
the future, for Civ1/Civ2 differences which are not covered by 
rulesets.

----------------------------------------------------------------------
The AI:
-------

An important caveat regarding rulesets is that while rulesets allow 
considerable flexibility, the AI has been designed and tested using 
Civ2 rules.  So the AI may not do as well with modified rules.

I've made some changes to the AI to try to take some of the ruleset
changes into account, and I've fudged the combat calculations to try 
to avoid core dumps, but in general you should not expect the AI to 
work as well with non-default options (where Civ1 is definitely 
included in "non-default").

Examples of problems which have not been fixed:
- Units not having hitpoints in Civ1 can make a big difference to 
  appropriate combat strategies.  
- City walls are sufficiently different (increased build cost, and 
  upkeep; and effect vs sea) to require different handling?
- Without the Harbour and Offshore Platform improvements, cities with
  many ocean squares are much more limited in Civ1.  
- Wonders which only work on a single continent are not given special
  consideration by the AI.

Hopefully this situation can be improved in the future, but the more 
flexible the rules, the harder to write a good AI...

----------------------------------------------------------------------
Implementation details:
-----------------------

This section and following section will be mainly of interested to
developers who are familiar with the freeciv source code.

Rulesets are mainly implemented in the server.  The server reads the
files, and then sends information to the clients.  Mostly rulesets 
are used to fill in the basic data tables on units etc, but in some
cases some extra information is required.

For units and advances, most of the information regarding each unit 
or advance is captured in the data tables, so it may be feasible to 
eventually make them fully customizable.  For improvements and 
wonders, each one has a largely unique effect, so the effects 
themselves are still hardwired in the code.  The "variant" field now 
allows some flexibility, though the effects themselves must still be 
hard-coded.

----------------------------------------------------------------------
Changes outline:
----------------

Server options: Server options can now optionally be string-valued.  
Three new string valued options to specify rulesets.

Loading rulesets: New files server/ruleset.[ch] to read in the 
ruleset files, and functions for sending the data to clients.  Moved 
datafilename from client/climisc to common/shared so the server can 
use it.  (And related changes to ser script and Makefiles.)

Communication: New and modified packets and related functions and 
cases.  (The game_info packet has been extended, which is ok because
capabilities ensure only ruleset-aware clients and servers are 
allowed to talk to each other.)

Variants: Added variant field to "struct improvement_type"; it gets 
set with the other fields when rulesets are loaded.  Scattered 
changes to implement the new variant effects.  Added a new widget in 
the help for wonders and improvements to show variant value, and new 
help text to describe each variant.

Roles: Added roles field to "struct unit_type".  Added new functions
in unit.c to make it easy to select units with given flag or give
role.  Made unit flag macros into an enumerated type which is only
converted into bit-masks internally.

Expanded "struct civ_game": Strings for the new server options, and 
miscellaneous ruleset information which doesn't go into the data
tables.

Obsoleted: Removed hardwired data tables for advances, unit_types, 
and improvement_types.  They are not considered useful as a fallback,
because then they may not be what the user wants.  
Removed set_civ_style() and related functions, since their effect
is now achieved by rulesets.

Remove hardwiring: various scattered changes to reduce cases of techs,
tech requirements, etc being hardwired, to reduce breakage if tables
are modified. 

Client-side continents:  Added functions to calculate continents from
the point of view of the client, for continent-based wonder effect 
variants.  If the client gets continents wrong (due to connecting 
tiles being unknown), it might for example let you build a power 
plant when it should be wonder-replaced by Hoover Dam.  But the 
effects will still be server-calculated, and hence take into account 
correct continents.
(I'm not sure if this is the best solution, but it's better (for
this) than the client having no idea about continents.  Maybe the
city_info packet should specify which improvements are obsolete or 
wonder-replaced...?)

Makefiles: Added new files to distribution; new server/ruleset.o.

New/convenience functions:
improvement_variant(), player_owns_active_govchange_wonder().

----------------------------------------------------------------------
TODO:
-----

- More improvement and wonder variants for Civ1 (see next section).
- Other alternative (non-Civ1) variants?  (If desired.)

- Could un-hardwire the advances required for: 
  bridges, railroad, fortress, government types.

- Could unhardwire effect of Mysticism on Temples;
  (Any other advances which affect building effects?)
- List of advances which affect trade routes revenue.
- List of advances which increase population-based pollution.

- Work on making the AI handle non-default cases better.
- Could have other information to help the AI?
  Eg, scores for units/techs/buildings, to help the AI know which 
  ones to prefer?

- Rulesets for: terrain, governments, ... ??

----------------------------------------------------------------------
TODO Variants: 
--------------

First, those variants which have been implemented:
(Note Variant=0 should always be "standard-freeciv" effect.)

Barracks:        Variant=0: just works for ground units (Civ2)
                 Variant=1: works also for air and sea units (Civ1)
Barracks II, Barracks III, Sun Tzu:
                 These use the same variant value and effect as
                 Barracks.
City Walls:      Variant=0: only applies against land and heli units
                 Variant=1: applies against sea units too
Police Station:  This uses the same variant value and effect as 
                 Women's Suffrage (see below).

Pyramids:        Variant=0: counts as Granary in every city (Civ2)
                 Variant=1: allows all government types and there is 
                            no transition anarchy (Civ1)
United Nations:  Variant=0: units regain extra hp per turn (freeciv)
                 Variant=1: allows all government types and there is 
                            no transition anarchy (Civ1) (see note(*))
Hoover Dam:      Variant=0: works on all cities owned
                 Variant=1: only works for cities on the same 
                            continent as where the wonder is built
J.S. Bach:       Variant=0: works on all cities owned
                 Variant=1: only works for cities on the same 
                            continent as where the wonder is built
Women's Suff.:   Variant=0: -1 unit-unhappy per City (-2 under 
                            Democracy) (freeciv)
                 Variant=1: -1 unit-unhappy per Unit (Civ1)
Magellan's Exp.: Variant=0: gives sea units 2 extra move points
                 Variant=1: gives sea units only 1 extra move point
Great Wall:      Uses same variant and effect as City Walls.

Other differences between Civ1 and Civ2/freeciv which have not been
implemented as variants:

Copernicus' Obs.: freeciv: science output +50% in a city
                  Civ1:    doubles knowledge production in city, after
                           all other effects.
Isaac Newton's:   freeciv: science output +100% in a city
                  Civ1:    "increases effects of Libraries and 
                           Universities" - by how much??
Michelangelo's:   freeciv: same as a Cathedral in every city
                  Civ1:     "increases effect of cathedrals" - by
                            how much??

There are other more minor Civ1/Civ2/freeciv differences, which 
could be handled using variants, for:
    Courthouse 
    Cure For Cancer
    Hanging Gardens
    Lighthouse

Notes:

Aqueduct and Sewer System: 
   The maximum sizes for cities without these improvements is handled 
separately, without using variants.  This leaves variants available 
if some other effects are desired.

Colosseum, Cathedral:
   Note that the technology effects on the effectiveness of Cathedrals
and Colosseums are not handled by variants, but by specially noting
the techs; see game.rtech fields cathedral_plus, cathedral_minus, 
and colosseum_plus.  Similar could be done for Temple and Mysticism 
(but there Civ1 is the same, and the AI calcs are complicated by 
the Oracle, which also affects Temples).

(*) United Nations: 
    I'm not sure what to do here: the diplomatic effect of Civ1,Civ2 
isn't appropriate for multiplayer freeciv, and the hp effect of 
freeciv isn't appropriate for Civ1 (no hitpoints).  The govchange 
effect seems reasonable -- note there is no Statue of Liberty in 
Civ1, and Pyramids, which has the same effect in Civ1, becomes 
obsolete.  Alternatively a Suffrage-like effect would seem sensible: 
your population is more willing to go to war if it has the sanction 
of the United Nations!

----------------------------------------------------------------------