[Freeciv-Dev] (PR#8164) building out of a builddir: generate_packets.py

["Jason Short" <jdorje@xxxxxxxxxxxxxxxxxxxxx>]

<URL: http://rt.freeciv.org/Ticket/Display.html?id=8164 >

If you take a pristine tarball and run

   tar xzf freeciv-xxx.tar.gz
   mkdir build
   cd build
   ../freeciv-xxx/configure
   make

you'll build Freeciv using a builddir separate from the srcdir.  In this 
case the srcdir is freeciv-xxx and the builddir is build.

However if you try to do this with a cvs checkout you run into two problems:

- Autogen won't work.  You can't do ../freeciv-xxx/autogen.sh; it 
refuses to run.  Nor can you easily do autogen.sh in the srcdir before 
changing to be builddir because autogen.sh the runs configure out of the 
srcdir (which means you'll need to "make distclean" before continuing). 
   Autogen.sh should be changed so that it will work if run out of the 
builddir.  (All of the files generated by autogen should be put in the 
srcdir.)

- generate_packets.py won't work.  So long as it doesn't need to be run 
this isn't a problem.  But if you edit packets.def you'll end up with a 
compilation that fails.  What should happen is that if generate_packets 
doesn't need to be run, the _gen files out of the srcdir are used.  But 
if generate_packets is run it should build the files in the builddir, 
and then they are used instead.

Why you would want to be able to do this is probably a question for the 
imagination.  The short answer is that since this is standard 
automake/autoconf behavior, it's something that we should support.  A 
few examples:

- You can have one srcdir (where you edit the code) and one builddir for 
each GUI.  Then when editing the code you can easily build the different 
clients without the need to reconfigure.  (I would like to be able to do 
this.)

- You can have a separate srcdir and builddir, and then when you edit 
packets.def the srcdir's copy of the _gen files won't be changed.  This 
is one way to avoid having the _gen files appear in your diffs.

-----

The attached patch fixes generate_packets.py to work when using a 
separate builddir.

The first problem is that generate_packets is run from the builddir, but 
is present in the srcdir.  Therefore we need to include a path.  We also 
need to tell generate_packets where to find packets.def (again in the 
srcdir).

This allows the ***_gen files to be built in the builddir.  Next we have 
to use the versions of the files in the builddir over the ones in the 
srcdir.  To do this we have to change the include directories. 
Fortunately automake already has a method for including a file from the 
builddir: by using <> instead of "" for the file name.  This changes the 
order of the directory search (for instance this is done with 
<config.h>, which is present in the builddir not the srcdir).  Finally 
we have to add to the INCLUDES so that the correct builddir directories 
are included.  . and $(top_builddir) are already included as part of 
DEFAULT_INCLUDES.  Therefore we have to add (depending on the directory) 
$(top_builddir)/common, $(top_builddir)/server, and 
$(top_builddir)/client to INCLUDES.  (Perhaps there is another way to do 
this, but I don't know what it is.)

As a side note, I don't think the files in client/gui-xxx/ should 
include packhand_gen.h or packets_gen.h.  But since they do we have to 
edit all the gui directories as well.

The end result is that things work.  You can test it by editing 
packets.def and seeing your changes echoed in the builddir only.  I've 
put it through pretty extensive testing, and except in some corner cases 
(there's at least one place where srcdir and builddir are still used 
interchangably) it works as it should.

jason

Index: ai/Makefile.am
===================================================================
RCS file: /home/freeciv/CVS/freeciv/ai/Makefile.am,v
retrieving revision 1.15
diff -u -r1.15 Makefile.am
--- ai/Makefile.am      21 Sep 2003 14:02:14 -0000      1.15
+++ ai/Makefile.am      17 Mar 2004 19:00:29 -0000
@@ -2,7 +2,7 @@
 
 noinst_LIBRARIES = libcivai.a
 
-INCLUDES = -I$(srcdir)/../common -I$(srcdir)/../server -I../intl 
-I$(top_srcdir)/common/aicore
+INCLUDES = -I../common -I../server -I$(srcdir)/../common -I$(srcdir)/../server 
-I../intl -I$(top_srcdir)/common/aicore
 
 ## Above, note -I../intl instead of -I$(top_srdir/intl) is deliberate.
 
Index: client/Makefile.am
===================================================================
RCS file: /home/freeciv/CVS/freeciv/client/Makefile.am,v
retrieving revision 1.53
diff -u -r1.53 Makefile.am
--- client/Makefile.am  6 Dec 2003 19:23:50 -0000       1.53
+++ client/Makefile.am  17 Mar 2004 19:00:30 -0000
@@ -129,7 +129,7 @@
 
 bin_PROGRAMS = civclient
 
-INCLUDES = -I$(srcdir)/include -I$(top_srcdir)/common 
-I$(top_srcdir)/common/aicore -I../intl -I$(srcdir)/agents @SOUND_CFLAGS@
+INCLUDES = -I../common -I$(srcdir)/include -I$(top_srcdir)/common 
-I$(top_srcdir)/common/aicore -I../intl -I$(srcdir)/agents @SOUND_CFLAGS@
 
 ## Above, note -I../intl instead of -I$(top_srdir/intl) is deliberate.
 
Index: client/packhand.h
===================================================================
RCS file: /home/freeciv/CVS/freeciv/client/packhand.h,v
retrieving revision 1.37
diff -u -r1.37 packhand.h
--- client/packhand.h   28 Nov 2003 17:37:19 -0000      1.37
+++ client/packhand.h   17 Mar 2004 19:00:30 -0000
@@ -15,7 +15,7 @@
 
 #include "map.h"
 
-#include "packhand_gen.h"
+#include <packhand_gen.h>
 
 void notify_about_incoming_packet(struct connection *pc,
                                   int packet_type, int size);
Index: client/agents/Makefile.am
===================================================================
RCS file: /home/freeciv/CVS/freeciv/client/agents/Makefile.am,v
retrieving revision 1.2
diff -u -r1.2 Makefile.am
--- client/agents/Makefile.am   21 Dec 2002 14:19:05 -0000      1.2
+++ client/agents/Makefile.am   17 Mar 2004 19:00:30 -0000
@@ -2,7 +2,7 @@
 
 noinst_LIBRARIES = libagents.a
 
-INCLUDES = -I. -I$(srcdir)/.. -I$(top_srcdir)/common/aicore 
-I$(srcdir)/../include -I$(top_srcdir)/common -I../../intl 
-I$(srcdir)/../gui-gtk @CLIENT_CFLAGS@
+INCLUDES = -I../../common -I.. -I$(srcdir)/.. -I$(top_srcdir)/common/aicore 
-I$(srcdir)/../include -I$(top_srcdir)/common -I../../intl 
-I$(srcdir)/../gui-gtk @CLIENT_CFLAGS@
 
 libagents_a_SOURCES =          \
        agents.c                \
Index: client/gui-gtk/Makefile.am
===================================================================
RCS file: /home/freeciv/CVS/freeciv/client/gui-gtk/Makefile.am,v
retrieving revision 1.17
diff -u -r1.17 Makefile.am
--- client/gui-gtk/Makefile.am  23 Feb 2003 14:11:14 -0000      1.17
+++ client/gui-gtk/Makefile.am  17 Mar 2004 19:00:30 -0000
@@ -1,7 +1,7 @@
 ## Process this file with automake to produce Makefile.in
 
 noinst_LIBRARIES = libguiclient.a
-INCLUDES = -I. -I$(srcdir)/.. -I$(srcdir)/../include -I$(top_srcdir)/common 
-I$(top_srcdir)/common/aicore -I../../intl -I$(srcdir)/../agents @CLIENT_CFLAGS@
+INCLUDES = -I.. -I../../common -I$(srcdir)/.. -I$(srcdir)/../include 
-I$(top_srcdir)/common -I$(top_srcdir)/common/aicore -I../../intl 
-I$(srcdir)/../agents @CLIENT_CFLAGS@
 
 ## Above, note -I../../intl instead of -I$(top_srdir/intl) is deliberate.
 # The INCLUDES "-I." is so resources.c includes the locally generated 
Index: client/gui-gtk-2.0/Makefile.am
===================================================================
RCS file: /home/freeciv/CVS/freeciv/client/gui-gtk-2.0/Makefile.am,v
retrieving revision 1.4
diff -u -r1.4 Makefile.am
--- client/gui-gtk-2.0/Makefile.am      23 Feb 2003 14:11:14 -0000      1.4
+++ client/gui-gtk-2.0/Makefile.am      17 Mar 2004 19:00:30 -0000
@@ -1,7 +1,7 @@
 ## Process this file with automake to produce Makefile.in
 
 noinst_LIBRARIES = libguiclient.a
-INCLUDES = -I. -I$(srcdir)/.. -I$(srcdir)/../include -I$(top_srcdir)/common 
-I$(top_srcdir)/common/aicore -I../../intl -I$(srcdir)/../agents @CLIENT_CFLAGS@
+INCLUDES = -I.. -I../../common -I$(srcdir)/.. -I$(srcdir)/../include 
-I$(top_srcdir)/common -I$(top_srcdir)/common/aicore -I../../intl 
-I$(srcdir)/../agents @CLIENT_CFLAGS@
 
 ## Above, note -I../../intl instead of -I$(top_srdir/intl) is deliberate.
 # The INCLUDES "-I." is so resources.c includes the locally generated 
Index: client/gui-sdl/Makefile.am
===================================================================
RCS file: /home/freeciv/CVS/freeciv/client/gui-sdl/Makefile.am,v
retrieving revision 1.6
diff -u -r1.6 Makefile.am
--- client/gui-sdl/Makefile.am  8 May 2003 03:24:44 -0000       1.6
+++ client/gui-sdl/Makefile.am  17 Mar 2004 19:00:30 -0000
@@ -1,7 +1,7 @@
 ## Process this file with automake to produce Makefile.in
 
 noinst_LIBRARIES = libguiclient.a
-INCLUDES = -I. -I$(srcdir)/.. -I$(srcdir)/../include -I$(top_srcdir)/common 
-I$(top_srcdir)/common/aicore -I../../intl -I$(srcdir)/../agents @CLIENT_CFLAGS@
+INCLUDES = -I.. -I../../common -I$(srcdir)/.. -I$(srcdir)/../include 
-I$(top_srcdir)/common -I$(top_srcdir)/common/aicore -I../../intl 
-I$(srcdir)/../agents @CLIENT_CFLAGS@
 
 ## Above, note -I../../intl instead of -I$(top_srdir/intl) is deliberate.
 # The INCLUDES "-I." is so resources.c includes the locally generated 
Index: client/gui-stub/Makefile.am
===================================================================
RCS file: /home/freeciv/CVS/freeciv/client/gui-stub/Makefile.am,v
retrieving revision 1.3
diff -u -r1.3 Makefile.am
--- client/gui-stub/Makefile.am 23 Feb 2003 14:11:14 -0000      1.3
+++ client/gui-stub/Makefile.am 17 Mar 2004 19:00:30 -0000
@@ -1,7 +1,7 @@
 ## Process this file with automake to produce Makefile.in
 
 noinst_LIBRARIES = libguiclient.a
-INCLUDES = -I. -I$(srcdir)/.. -I$(srcdir)/../include -I$(top_srcdir)/common 
-I../../intl @CLIENT_CFLAGS@
+INCLUDES = -I.. -I../../common -I$(srcdir)/.. -I$(srcdir)/../include 
-I$(top_srcdir)/common -I../../intl @CLIENT_CFLAGS@
 
 ## Above, note -I../../intl instead of -I$(top_srdir)/intl is deliberate.
 
Index: client/gui-win32/Makefile.am
===================================================================
RCS file: /home/freeciv/CVS/freeciv/client/gui-win32/Makefile.am,v
retrieving revision 1.8
diff -u -r1.8 Makefile.am
--- client/gui-win32/Makefile.am        9 Nov 2003 16:45:01 -0000       1.8
+++ client/gui-win32/Makefile.am        17 Mar 2004 19:00:30 -0000
@@ -2,7 +2,7 @@
 
 noinst_LIBRARIES = libguiclient.a
 
-INCLUDES = -I. -I$(srcdir)/.. -I$(srcdir)/../include 
-I$(top_srcdir)/common/aicore -I$(top_srcdir)/common -I../../intl 
@CLIENT_CFLAGS@
+INCLUDES = -I.. -I../../common -I$(srcdir)/.. -I$(srcdir)/../include 
-I$(top_srcdir)/common/aicore -I$(top_srcdir)/common -I../../intl 
@CLIENT_CFLAGS@
 
 ## Above, note -I../../intl instead of -I$(top_srdir)/intl is deliberate.
 
Index: client/gui-xaw/Makefile.am
===================================================================
RCS file: /home/freeciv/CVS/freeciv/client/gui-xaw/Makefile.am,v
retrieving revision 1.14
diff -u -r1.14 Makefile.am
--- client/gui-xaw/Makefile.am  28 Jul 2003 01:11:48 -0000      1.14
+++ client/gui-xaw/Makefile.am  17 Mar 2004 19:00:30 -0000
@@ -1,7 +1,7 @@
 ## Process this file with automake to produce Makefile.in
 
 noinst_LIBRARIES = libguiclient.a
-INCLUDES = -I. -I$(srcdir)/.. -I$(srcdir)/../include -I$(top_srcdir)/common 
-I../../intl -I$(top_srcdir)/common/aicore -I../../intl -I$(srcdir)/../agents 
@CLIENT_CFLAGS@
+INCLUDES = -I.. -I../../common -I$(srcdir)/.. -I$(srcdir)/../include 
-I$(top_srcdir)/common -I../../intl -I$(top_srcdir)/common/aicore -I../../intl 
-I$(srcdir)/../agents @CLIENT_CFLAGS@
 
 ## Above, note -I../../intl instead of -I$(top_srdir/intl) is deliberate.
 # The INCLUDES "-I." is so resources.c includes the locally generated 
Index: common/Makefile.am
===================================================================
RCS file: /home/freeciv/CVS/freeciv/common/Makefile.am,v
retrieving revision 1.49
diff -u -r1.49 Makefile.am
--- common/Makefile.am  13 Feb 2004 07:57:58 -0000      1.49
+++ common/Makefile.am  17 Mar 2004 19:00:30 -0000
@@ -99,4 +99,4 @@
 
 BUILT_SOURCES = packets_gen.c packets_gen.h
 packets_gen.h packets_gen.c: packets.def generate_packets.py
-       ./generate_packets.py
+       $(top_srcdir)/common/generate_packets.py $(top_srcdir)
Index: common/generate_packets.py
===================================================================
RCS file: /home/freeciv/CVS/freeciv/common/generate_packets.py,v
retrieving revision 1.7
diff -u -r1.7 generate_packets.py
--- common/generate_packets.py  15 Jan 2004 11:52:38 -0000      1.7
+++ common/generate_packets.py  17 Mar 2004 19:00:30 -0000
@@ -1323,7 +1323,10 @@
 # various files.
 def main():
     ### parsing input
-    input_name="packets.def"
+    if (len(sys.argv) < 2):
+        input_name="packets.def"
+    else:
+        input_name="%s/common/packets.def"%sys.argv[1]
     content=open(input_name).read()
     content=strip_c_comment(content)
     lines=string.split(content,"\n")
@@ -1516,7 +1519,7 @@
 
 #include "packets.h"
 
-#include "hand_gen.h"
+#include <hand_gen.h>
     
 bool server_handle_packet(enum packet_type type, void *packet,
                          struct player *pplayer, struct connection *pconn)
@@ -1567,7 +1570,7 @@
 
 #include "packets.h"
 
-#include "packhand_gen.h"
+#include <packhand_gen.h>
     
 bool client_handle_packet(enum packet_type type, void *packet)
 {
Index: common/packets.h
===================================================================
RCS file: /home/freeciv/CVS/freeciv/common/packets.h,v
retrieving revision 1.162
diff -u -r1.162 packets.h
--- common/packets.h    14 Jan 2004 11:58:12 -0000      1.162
+++ common/packets.h    17 Mar 2004 19:00:30 -0000
@@ -65,7 +65,7 @@
   AUTH_NEWUSER_RETRY  /* inform the client to try a different [new] password */
 };
 
-#include "packets_gen.h"
+#include <packets_gen.h>
 
 void *get_packet_from_connection(struct connection *pc, enum packet_type 
*ptype, bool *presult);
 void remove_packet_from_buffer(struct socket_packet_buffer *buffer);
Index: server/Makefile.am
===================================================================
RCS file: /home/freeciv/CVS/freeciv/server/Makefile.am,v
retrieving revision 1.32
diff -u -r1.32 Makefile.am
--- server/Makefile.am  6 Dec 2003 19:23:51 -0000       1.32
+++ server/Makefile.am  17 Mar 2004 19:00:31 -0000
@@ -4,7 +4,7 @@
 
 bin_PROGRAMS = civserver
 noinst_LIBRARIES = libcivserver.a
-INCLUDES = -I$(srcdir)/../common -I$(srcdir)/../ai -I../intl 
-I$(top_srcdir)/common/aicore -I$(srcdir)/userdb
+INCLUDES = -I../common -I$(srcdir)/../common -I$(srcdir)/../ai -I../intl 
-I$(top_srcdir)/common/aicore -I$(srcdir)/userdb
 
 ## Above, note -I../intl instead of -I$(top_srdir/intl) is deliberate.
 
Index: server/cityhand.h
===================================================================
RCS file: /home/freeciv/CVS/freeciv/server/cityhand.h,v
retrieving revision 1.29
diff -u -r1.29 cityhand.h
--- server/cityhand.h   14 Feb 2004 02:21:25 -0000      1.29
+++ server/cityhand.h   17 Mar 2004 19:00:31 -0000
@@ -18,7 +18,7 @@
 struct connection;
 struct conn_list;
 
-#include "hand_gen.h"
+#include <hand_gen.h>
 
 void really_handle_city_sell(struct player *pplayer, struct city *pcity,
                             Impr_Type_id id);
Index: server/diplhand.h
===================================================================
RCS file: /home/freeciv/CVS/freeciv/server/diplhand.h,v
retrieving revision 1.8
diff -u -r1.8 diplhand.h
--- server/diplhand.h   19 Feb 2004 21:06:42 -0000      1.8
+++ server/diplhand.h   17 Mar 2004 19:00:31 -0000
@@ -18,7 +18,7 @@
 struct packet_diplomacy_info;
 struct connection;
 
-#include "hand_gen.h"
+#include <hand_gen.h>
 
 void establish_embassy(struct player *pplayer, struct player *aplayer);
 
Index: server/maphand.h
===================================================================
RCS file: /home/freeciv/CVS/freeciv/server/maphand.h,v
retrieving revision 1.39
diff -u -r1.39 maphand.h
--- server/maphand.h    28 Nov 2003 17:37:22 -0000      1.39
+++ server/maphand.h    17 Mar 2004 19:00:31 -0000
@@ -17,7 +17,7 @@
 #include "packets.h"
 #include "terrain.h"
 
-#include "hand_gen.h"
+#include <hand_gen.h>
 
 enum ocean_land_change { OLC_NONE, OLC_OCEAN_TO_LAND, OLC_LAND_TO_OCEAN };
 
Index: server/plrhand.h
===================================================================
RCS file: /home/freeciv/CVS/freeciv/server/plrhand.h,v
retrieving revision 1.62
diff -u -r1.62 plrhand.h
--- server/plrhand.h    31 Jan 2004 17:52:42 -0000      1.62
+++ server/plrhand.h    17 Mar 2004 19:00:31 -0000
@@ -19,7 +19,7 @@
 #include "packets.h"
 #include "shared.h"            /* fc__attribute */
 
-#include "hand_gen.h"
+#include <hand_gen.h>
 
 struct player;
 struct section_file;
Index: server/unithand.h
===================================================================
RCS file: /home/freeciv/CVS/freeciv/server/unithand.h,v
retrieving revision 1.36
diff -u -r1.36 unithand.h
--- server/unithand.h   28 Nov 2003 17:37:22 -0000      1.36
+++ server/unithand.h   17 Mar 2004 19:00:31 -0000
@@ -16,7 +16,7 @@
 #include "packets.h"
 #include "unit.h"
 
-#include "hand_gen.h"
+#include <hand_gen.h>
 
 bool handle_unit_move_request(struct unit *punit, int dest_x, int dest_y,
                             bool igzoc, bool move_diplomat_city);