[Freeciv-Dev] Re: [RFC][Patch] Reduce bandwith by using deltas

["Per I. Mathisen" <per@xxxxxxxxxxx>]

On Sat, 2 Nov 2002, Raimar Falke wrote:
> > Surely there must be a way to do this cleaner through some macro
> > magic?
>
> I thought about it and came to the conclusion: no. The basic problem
> is that we need the list of fields multiple times: in the struct
> definition, the send function and the receive function. With normal
> macros we can only generate one of these. We have to have some kind of
> memory for the preprocessor. While this may be possible I don't know
> how to do this.

Ah, I knew my experience with XSLT would come handy one day. You can solve
any kind of text parsing quite nicely without memory if you can use
recursion, as long as you keep the logical structure intact.

You can define packets in, say, packets.h.in, like this

        PACKET_START(packet_player_info)
        DATA_STRING(name, MAX_LEN_NAME)
        DATA_BOOL(is_name)
        CAPABILITY("team", DATA_UNIT8(team), team = TEAM_NONE)
        DATA_UNIT8(government)
        ...
        PACKET_END

Then packets.h like this

        #define PACKET_START(x) struct x {
        #define DATA_STRING(x, len) char x[len];
        ...
        #define PACKET_END };
        #include "packets.h.in"

Then packets.c like this

        #include "packets.h"

        #undef PACKET_START
        #define PACKET_START(x) \
                int send_x(struct connection *pc, \
                           const struct x *packet) \ { ...
        #undef DATA_STRING
        #define DATA_STRING(x, len) \
                dio_put_string(&dout, packet->x); /* len */
        ...
        #include "packets.h.in"

        #undef PACKET_START
        #define PACKET_START(x) \
                struct x *receive_x(struct connection *pc) \ { ...
        #undef DATA_STRING
        #define DATA_STRING(x, len) \
                dio_get_string(&din, packet->name, sizeof(packet->name)); /* 
len */
        ...
        #include "packets.h.in"

This will generate vanilla packets.c and packets.h without compile
dependendency on perl. Also packets*.c|h will be much smaller (from 150k
to maybe 2k), and you only need to update one file to add a new packet.
You can even autogenerate the packet_type enum this way.

The delta code is a little bit more complicated, so you will need to
restructure the logic of it, because the macros cannot change the logical
layout of the transformed text. One way is to change this

/* ... lots of BV_SET() ... */
+  if (!BV_ISSET_ANY(fields) && already_known) {
+    freelog(LOG_DEBUG, "  no change -> discard");
+    stats_packet_game_info_discarded++;
+    return 0;
+  }
/* ... lots of BV_ISSET() ... */

to doing all the work on one field at once, and instead discard the packet
after it has been fully assembled. This is cleanest, but will waste some
CPU compared to the above solution.

A way to keep the CPU savings is to make the generated function
recursive... in PACKET_START(x) you declare "static bool second_run =
FALSE; second_run = !second_run;" and in PACKET_END(x) you do "if
(second_run && BV_ISSET_ANY(fields)) second_run send_packet_x(packet);".
Then you make data defines like

        #undef DATA_STRING
        #define DATA_STRING(x, len) \
                if (!second_run) { ... BV_SET( code ) ... } \
                if (second_run) { ... BV_ISSET( code ) ... }

Now _please_ shoot this into flames if you think it is stupid before I
start any coding :)

  - Per