[Freeciv-Dev] RFC: Testing framework

[Per I Mathisen <per@xxxxxxxxxxx>]

This is a proof-of-concept testing framework for Freeciv. It loads a
savegame created with civworld and carries out a series of tests on it.
The tests are currently quite few and very simple, so as it is presented
here it isn't yet useful.

However, when we get ten times as many tests (and more), I think it should
start working nicely as a contribution towards making sure changes don't
break stuff.

It currently loads ruleset info from the data/default directory, which is
wrong. It should have its own rulesets, since we will want to test a lot
of strange and unusual units, techs, buildings, nations, etc..

Currently, when you do "(cd tests; make check)" it outputs this:

== Test run "COMMON" ==
Diplomat bribe unit ... ok
!Diplomat bribe unitless tile ... ok
Spy bribe unit ... ok
Spy embassy city ... ok
Spy steal tech from city ... ok
Spy investigate city ... ok
Spy is diplomat (sic) ... ok
Fighter is air unit ... ok
!Fighter is heli unit ... ok
!Fighter is ground unit ... ok
!Helicopter is air unit ... ok
Helicopter is heli unit ... ok
!Helicopter is ground unit ... ok

== Test run "SERVER" ==
Counting diplomats ... ok
City has barracks ... ok
Find closest city ... ok
City can work tile ... ok
City can place worker at tile ... ok
PASS: savegame
==================
All 1 tests passed
==================

To begin with, I will use this unit testing framework to prove that my
unit flags patch (which adds 6 new unit flags) does what it is supposed to
do, and ensure that other patches don't break this patch later on. I also
hope that the gen. impr. patch, for instance, can be proven to work this
way.

The advantages of such unit testing are
 - patch writer gets to test assumptions about the patch
 - maintainers get a way to see that the patch does what it is supposed to
   do and if patch writer has thought about all possibilities for failure
 - future patch writers get a way to see if their new patch breaks
   some rarely used features, breakage that won't show up in manual
   testing using default rulesets
 - less bugs, which means quicker releases :)

Comments?

Yours
Per

tests.tar.gz
Description: Binary data

tests.diff
Description: Text document