[freeciv-ai] Re: exploration madness

[Raimar Falke <rf13@xxxxxxxxxxxxxxxxx>]

On Tue, Feb 11, 2003 at 10:13:42PM +0000, Per I. Mathisen wrote:
> On Tue, 11 Feb 2003, Raimar Falke wrote:
> > On Tue, Feb 11, 2003 at 01:08:37PM +0000, Per I. Mathisen wrote:
> > > Yes, it is interesting. I take back my suspicion of the explorer code. I
> > > guess something is wrong with targeting code, so maybe I fucked up
> > > something in the ksw and fstk cleanups I committed.
> >
> > Old topic: AI measurement/metrics now.
> 
> Old question: How would you measure this (in a way that would catch the
> bug mentioned in this thread)?

For each patch which changes behavior a test case has to be provided
by the patch author for every behavior change. A test case consists of
a savegame and code which measures a certain metric. The patch has to
show an improvement in this metric.

For all patches (both of change-behavior and doesn't-change-behavior)
all test cases have to be evaluated. A change for a
doesn't-change-behavior patch is not acceptable. If a change-behavior
patch decreases the metric of a certain other test case the
maintainers have to weigh up.

In addition to the above you need some basic test cases like
time-to-republic, harvested-resources, number-of-cities, ...

In this thread multiple problems were mentioned. I just pick one:
conquer-speed. The defending player would have n cities. Each city has
a different number of defenders starting with 0 and increasing by
one. The defenders are fortified. The defending player is controlled
by a nop-AI. This AI will not built any units or improvements. It
won't move the units nor will it change the activity of the
units. This AI isn't currently in the code but I think it should be
possible to provide such an AI under a new AI mode called "useless"
;). The attacking player has also n cities but no units. It has the
choice over several unit types.

Now you have to close the loop holes: The two players are on one
island. All space is used by the cities (this is to make sure that the
player attacks and is not building settlers). No government change is
possible/allowed. No research is possible/research is too
expensive. Also some side conditions have to be taken care of. Such
as: the government should only cause no or only very mild upset by
units in the field.

The goal is to conquer the cities of the defending player as fast as
possible. The code has to measure this.

The description outlined above is a base case. The base case should be
as rigid as possible. There are a lot of possibilities to expand
it. For example you can give more land:

  ################      ########################
  ###a####a####a###########b####b####b##########
  ################      ########################

# - land
a - city of attacking player
b - city of defending player

Now the AI has to decide if it will first get the enemy cities or if
it will send a settler in the free land behind the cities.

        Raimar

-- 
 email: rf13@xxxxxxxxxxxxxxxxx
  The trick is to keep breathing.