[Freeciv-Dev] (PR#11068) savegame file compression

["Christian Knoke" <chrisk@xxxxxxxxx>]

<URL: http://rt.freeciv.org/Ticket/Display.html?id=11068 >


FWIW, some analysis on the replacement of the savegame file compression
method in Freeciv to bzip2.

Abstract:
On average, bzip2 -9 gives files of 60% size compared with gzip -6. But on
some files, it can cut size down to a quarter. The usage of bzip2 would
increase CPU time of civserver (at turn done, most relevant with AI) by
approx. 11%. The usage of gzip -9 is not an alternative, neither in size nor
in time.


On Wed, Nov 17, 2004 at 07:10:34PM -0800, Jason Short wrote:
> 
> > Both files are ~1,4MB
> 
> Seems like RT silently rejects any files that large.
> 
> However if I bzip it it is small enough.

Some years ago we've had a discussion about the compression of savefiles,
and at that time I was argueing, that a gzip -9 compressions slows things
down too much. But time has changed, Freeciv has, too.

I just made a quick test with 15 variant savefiles on my 2 GHz machine:

Method         Size        real time     per file
---------------------------------------------------------------
uncompressed  5349 KB
gzip -6        909 KB      0.321s        021 ms
gzip -9        869 KB      1.567s        104 ms
bzip2 -9       533 KB      1.194s        079 ms

So, bzip counts down the file size to 60% on average, and increases
compression time by factor 4.

With big files, like that one from Eric (32 nations, 210x140, 3899 units),
it looks like that:

Method         Size        per file
---------------------------------------------------------------
uncompressed  9560 KB
gzip -6       1384 KB      651 ms
gzip -9       1332 KB    4.639 ms
bzip2 -9       900 KB    1.705 ms

So, file size cut to 65%, time increase by factor 2.6.

With my biggest savegame here at hand (32 nations (15 dead), 80x50, 520
units):

Method         Size        per file
---------------------------------------------------------------
uncompressed  1416 KB
gzip -6        236 KB       87 ms
gzip -9        232 KB      380 ms
bzip2 -9        64 KB      375 ms

In practice, I see a cut like this (to 27%!) more often. Maybe somebody
wants to make his own tests?

And here, the output of  time civserver -f <file above> 
(just logging in, pressed turn done, logout, quit server):
1x turn done:  user    0m8.490s
2x turn done:  user    0m11.090s

That is, one turn in civserver, including gzip -6, takes 2.600 ms
Prognosted time with bzip2 -9:                           2.888 ms

which is an increase of 11% computing time, approximately. 

One turn in civclient GTK2 gives 110 ms in this case, BTW.

Christian

-- 
Christian Knoke            * * *            http://cknoke.de
* * * * * * * * *  Ceterum censeo Microsoft esse dividendum.