[Freeciv-Dev] Re: [RFC][Patch] Reduce bandwith by using deltas

[Raimar Falke <rf13@xxxxxxxxxxxxxxxxx>]

On Sat, Nov 02, 2002 at 02:28:11PM +0000, per wrote:
> On Fri, 1 Nov 2002, Raimar Falke wrote:
> > The idea is simple: instead of transferring all fields for a given
> > foo-info only the fields which have been changed are
> > transferred.
> 
> This is a brilliant idea. Like it lots.
> 
> But this got me thinking. Let's say we manage to do this in a clean and
> extensible way. 

> We add the possibility in dataio.c to output to file instead of to a
> network socket.

This is very easy since dataio works on memory blocks and not on
sockets.

> We use dataio, libz and good sense to keep
> the output small. Now we send packets to this file each turn as if it was
> an omniscient observer... what do we have?
> 
> A journal!
> 
> >From this we can do game replay or load a savegame from any given turn...

We need to filter the packet stream for each connection. In others
words we have to save the packet stream for each connection (if the
same packet goes to multiple connections it can be merged but these
are details).

> Now, I am not so happy about the implementation.

As expected.

> Surely there must be a way to do this cleaner through some macro
> magic?

I thought about it and came to the conclusion: no. The basic problem
is that we need the list of fields multiple times: in the struct
definition, the send function and the receive function. With normal
macros we can only generate one of these. We have to have some kind of
memory for the preprocessor. While this may be possible I don't know
how to do this.

> Also, all packets should be sent this way, not just some. The
> implementation should be sane and consistent.

As I said the delta-compression will only work for info packets. On
the other side we can generate the send/receive functions for the
remaining packets in the same way. No problem.

> > This delta-protocol is only possible for info packets. So for example
> > you don't want that your attack command is discarded because it looks
> > like the previous one.
> 
> Right, you can't _discard_ the packet, but you can reduce it in size by
> only sending a delta... Although the gain is likely to be very small, I
> think it is important to keep a consistent API.

Yes you are right.

> > The generator script is written in python. I could have used perl but
> > then it would have required multiple days to write it. Feel free to
> > convert it.
> 
> I have no preference for perl over python, though I don't really like
> generator scripts in the first place... but perhaps it is possible to
> unite them into one script?

If the script generates all packets: yes possible.

        Raimar

-- 
 email: rf13@xxxxxxxxxxxxxxxxx
 Q:  Do you know what the death rate around here is?
 A:  One per person.