[Freeciv-Dev] Re: Network bandwidth, and other things.

[Raimar Falke <rf13@xxxxxxxxxxxxxxxxx>]

On Thu, Nov 07, 2002 at 08:06:29PM +0100, Juan Uriarte wrote:
> > > > Also till now I didn't know a good way to merge the
> > > > packets in chunks (to yield better compression ratio). Using
> > > > PACKET_PROCESSING_{STARTED,FINISHED} is nice.
> > >
> > >  That is nice for the 'during the turn' packets, which get those 6 bytes
> > > of easily compressable data (0/3/3 and 0/3/2, IIRC).
> >
> > 6 bytes?
> 
>  Yeah, zlib.h says to have buffers of at least 12+1.01*original_size on
> compression for the worst case. My incorrect conclusion is that there is a
> 12 byte header and a worst case compression ratio of 101%.

Ok till here.

> Having those 6 bytes for every compression chunk 

You lost me here. Why 6?
 
> means we'll never hit the 101% case
> for small packets, so that leaves only packets with size less than approx.
> 8 bytes giving a >100% compression ratio. Of course this is utter voodoo
> and wild guessing from my part.
> 
> > > Readjusting some flags on SHORT_CITY_INFO could improve compression,
> > > the patch prints some flag statistics, to help improve the ratio.
> > > Actually a 32 bit flag field is sent always, with an optional 8bit
> > > flag field. Maybe splitting this into 16/32 would give better
> > > average packet size, but a careful study of flag usage is needed. I
> > > doubt there is much of a point, with zlib on top of it.
> >
> > There is another way to encode which fields have changed and so are
> > transfered: instead of using the bit i as a flag you send the number
> > i. So if we have
> 
> TLV (Type/Length/Value) encoding with variable size T. Next we'll be
> implementing it all in ASN.1 and BER. <duck> We could rip the code off
> the net-snmp package ... </duck>

BER? Does net-snmp have a generator? Which can do deltas?

> > packet_tile_info; PACKET_TILE_INFO
> >         COORD x,y; key
> >
> >         UINT8 type, known;
> >         UINT16 special;
> > end
> >
> > we need a value for type, known, special and one for
> > END-OF-HEADER. These are four values and we need 2bits for each. This
> > gives a variable header size in addition to the already variable body
> > size of the packet.
> >
> >   size_for_fixed = no_of_fields
> >   size_for_variable = log_2(no_of_fields + 1) * (number_of_fields_changed + 
> > 1)
> >
> > If no_of_fields = 15 we get:
> >   size_for_fixed = 15 bits
> >   size_for_variable = 4 + 4 * number_of_fields_changed
> >   variable is better if number_of_fields_changed <= 2
> >
> > If no_of_fields = 31 we get:
> >   size_for_fixed = 31 bits
> >   size_for_variable = 5 + 5 * number_of_fields_changed
> >   variable is better if number_of_fields_changed <= 5
> 
> number_of_fields_changed == sum of 1-bits in my implementation's fixed
> headers (Quersumme).

Also known as Hamming weight. See include/linux/bitops.h in your
linux-kernel sources.

> I'll see if I find some library function to calculate that value and
> keep a count history in the packet stats.

        Raimar

-- 
 email: rf13@xxxxxxxxxxxxxxxxx
 1 + 1 = 3, for large values of 1