concatenated block serialization

[rdojdk at arcor.de]

Hi!

A few years ago we had the problem that loading levels into the 3D game we where developing (in C++ but this part was plain C) was way to slow. There where just by far too many small objects that had to be loaded. We came up with a solution that I think might be applicable to serialization in the Java platform as well, although mainly for RMI and not so much for startup times.

We wrote a tool that first loaded those small objects into one continuous block of memory. Second it made all pointers relative to the start of this memory block (effectively rendering them invalid) and recorded the address of those pointers in a list. Finally it wrote the block and the list to a file.

Now loading was a snap: First we loaded the list into one block and the data block into another. Second we iterated over the list and added these pointer offsets to the base address of the data block to find the addresses of the pointers. Third we added the base address of the data block to them. This restored a completely working data block and we could release the pointer list block. The performance increase was stunning.

In the Java world the JVM would have to do all of this itself. Possibly for all Serializable objects... and in fact it could be that it already does, making this post quite embarrassing for me ;-) If Java bytecode allowed for relative references, things where even easier!

The main problem that I can see is about how to store the references to the classes that the objects are instances of. A straight forward solution would be to have a third block stored to disk: A map of class names to a list of these pointers' offsets. When deserializing, another loop would resolve the classes' address (maybe loading the class) and then assign that address to all pointers in the according list.

BTW: Especially mobile devices could have flash memory mapped into address space. When data is stored there, then it may even be possible to use the data block directly: The relative pointers can already be correct as well as references to classes that are stored there as well. Actually there would be almost no deserialization any more!

I would really appreciate any comment! Even if you think it's plain ridiculous or just the wrong list (I didn't find any more appropriate).


Kind regards!
  Rüdiger zu Dohna

Jetzt komfortabel bei Arcor-Digital TV einsteigen: Mehr Happy Ends, mehr Herzschmerz, mehr Fernsehen! Erleben Sie 50 digitale TV Programme und optional 60 Pay TV Sender, einen elektronischen Programmführer mit Movie Star Bewertungen von TV Movie. Außerdem, aktuelle Filmhits und prickelnde Erotik in der Arcor-Videothek. Infos unter www.arcor.de/tv