[Freeciv-Dev] Re: (PR#8662) [PATCH] tests/Makefile.am fixes

["Marko Lindqvist" <marko.lindqvist@xxxxxxxxxxx>]

<URL: http://rt.freeciv.org/Ticket/Display.html?id=8662 >

Jason Short wrote:

> 
> So does that mean your original patch is correct?

  Maybe :)

  For now both noinst_SCRIPTS and check_SCRIPTS work, but I don't know 
which will be right solution ultimately. Read on.

  Oh, I claimed that patch fixes running tests from builddir. Well, I 
changed commandline to run them, but I never looked into scripts 
themselves. They may not work as expected when started from builddir. 
Will Raimar as original script author look into this?

> What is needed for a full "fix" for automake "make check" correctness?

  Please note that I do not have first hand experience about doing these 
things. Anyhow, _if_ we decide to make current tests automake style 
"make check" tests, actual scripts should be changed so they either fail 
or success (AFAIK there is also special return code for tests that make 
no sense in current platform, but there is no platform specific tests in 
freeciv). They should be listed in check_SCRIPTS, meaning:
  - they are depencency for target 'check'
  - distributed, but not installed
  - automake is not supposed to know how to compile them

  They should be also listed in TESTS so automake runs them for target 
'check'. I guess that as long as all scripts are independent (there is 
no scripts that are just helpers for actual test scripts) we can simply 
"TESTS=$(check_SCRIPTS)".

  So, this seems to rather trivial change after all.


  BUT

  I'm not sure if we should run current tests for "make check". I think 
that target "check" is meant to make sure that whaterver "make all" 
produces is valid, not to analyze sources and such.  Then again, I don't 
know if running these tests with "make check" has any actual downsides. 
Maybe it confuses some users, maybe not.
  Biggest potential problem I can see is that maybe there is tools to 
automatically compile and install distributed packages, which does 'make 
check' and if package seems to be broken, aborts. We don't want such end 
user tools to fail simply because some sourcefile does not fully comply 
to freeciv style.
  So, we may want to make different target altogether for current tests. 
Biggest problem I see with that is that if we want such "make 
sourcetest" to work from top builddir, we need to define it in all 
makefiles (there might be way to avoid this, but I'm not aware of such 
make feature). Also, "make check" is target provided by many packets and 
as such people might test it even if they don't know if it works in 
freeciv. Nobody tries to make targets specific to freeciv without 
reading about it first.

  I try to look if there is some target widely used for tests like this 
(I might found out that "check" is typically used for this). Maybe even 
automake provides some support for this, after all.

  Opinions?


  - Caz