[Freeciv-Dev] (PR#11028) error on loading savegame

["Jason Short" <jdorje@xxxxxxxxxxxxxxxxxxxxx>]

<URL: http://rt.freeciv.org/Ticket/Display.html?id=11028 >

> [Erki.Savisaar@xxxxxx - Thu Nov 18 00:41:35 2004]:
> 
> > "These two"?  You forgot attachments.
> No I didn't forget. I looked my Sent Items folder and they are there.
> Maybe some virus software or something removed them during the trip...
> 
> I included these attachments also in this mail. In case they disappear
> again you can download them from URLs below:
> http://savisaar.ee/civgame+1812m.sav.gz
> http://savisaar.ee/civgame+1814.sav.gz 
> 
> Both files are ~1,4MB

Seems like RT silently rejects any files that large.

However if I bzip it it is small enough.  I attached it to the ticket as
a comment.

-----

The problem is the savegame is (unzipped) 10 megabytes in size.  This is
 66k lines comprising 400k "entries" (I'm not entirely sure what an
entry is).  Each entry requires a bucket in the hash table and for
efficiency the hash table is sparse.  So there are 1.5M hash table
buckets, each of which is 16 bytes.  This makes for a 25 megabyte hash
table.

The next question, of course, is what we can do about it.

The savegame was saved with code that had a buglet in it (since fixed)
causing units not always to be saved in a tabular format.  However when
I load and resave there is only a tiny difference (<<1%) in overall size
and no difference (I think) in the number of entries.  So this is no help.

On a 64-bit system the problem is even worse.  Here I think the hash
bucket will be 32 bytes in size so this savegame will take 50 megabytes
of memory to load.  With reordering of the struct (or maybe without,
depending on the alignment requirements) this could be dropped to 24 bytes.

And this is just the size of the hash buckets.  The data in the hash
will probably be much larger.  But that is done in a bunch (400k) small
malloc calls, so there are no red flags thrown up.

jason