[Freeciv-Dev] Re: It is feature freeze time

[Raimar Falke <hawk@xxxxxxxxxxxxxxxxxxxxxxx>]

On Mon, Apr 29, 2002 at 09:20:35AM +0200, Per I. Mathisen wrote:
> On Sun, 28 Apr 2002, Ross W. Wetmore wrote:
> > Will there be a standard set of test procedures that evolve to insure
> > that the result is usable and key elements aren't broken?
> 
> This, I think, is much warranted. I've been thinking of how to develop
> such a thing. The best way to do it is probably to make a pseudo-server
> that loads a constructed savegame and runs a series of tests on it.
> 
> Each function should have a few tests, for example
> 
>       FC_TEST(is_diplomat_on_tile(17,15), "diplomat was not found");
>       FC_TEST(!is_diplomat_on_tile(1,1), "diplomat found where it "\
>                                          "should not have been found");
> 
> printing on errors,
> 
>       FAILURE in test: is_diplomat_on_tile(17,15)
>       Message given: diplomat was not found

Either I misunderstood it or it doesn't make sense. If we:
 1) construct a savegame
 2) let a pseudo-server load the savegame and test various parts
there no way the constructed savegame can be modified. What could work
is something like this:
 1) construct a savegame
 2) let a real server load the game and let it run for while
 3) let a pseudo-server load the savegame and test various parts

Problem here is that in 2) you can't restrict the server (say only do
diplomacy actions or do only fiddle with your tech tree). So while you
may fiddle with the diplomacy code (and add a test code for this) the
city calculation is changed (say by the generalized improvement stuff)
and now the AI will choose different diplomacy actions. Or to say it
another way: we currently have only the server AI which will stimulate
various regions of code (and not even all, network for example). And
the AI as a "stimulator" is just too random/changes too easy.

Don't get me wrong I also would like a testing environment but the
above is very fragile. One possible solution may be so called unit
tests. However freeciv is currently too interweaved for such tests.

        Raimar

-- 
 email: rf13@xxxxxxxxxxxxxxxxx
  A supercomputer is a computer running an endless loop in just a second