LZMA compression for jpkg

[Chris Hegarty]

Hi,

I have been looking into the possibility of using LZMA compression ( 
following on from Dalibors work last year ) for the jmod packages. 
Initial results look positive.

I've been using a Java implementation from the latest LZMA 9.20 SDK [1]. 
I modified jpkg to use LZMA compression for classes, resources, and 
nativelibs. Here are two of the largest packages:

    > : du -sk jmod/jdk.boot at 7-ea.jmod jmod/sun.desktop at 7-ea.jmod
    12170    jmod/jdk.boot at 7-ea.jmod
    8584    jmod/sun.desktop at 7-ea.jmod
    > : du -sk jmod-lzma/jdk.boot at 7-ea.jmod jmod-lzma/sun.desktop at 7-ea.jmod
    8456    jmod-lzma/jdk.boot at 7-ea.jmod
    6791    jmod-lzma/sun.desktop at 7-ea.jmod

~30% reduction on jdk.boot and ~20% reduction for sun.desktop.

Here is a total for all the JDK modules:

     > : du -sk jmod jmod-lzma
     31171    jmod
     23936    jmod-lzma

~23% reduction in size .

Q: The results look good, but can we do better?

The debian packages that get built during the modules build are still 
smaller than these lzma-ized jmod files. The main difference seems to be 
that the deb packages are generated with best compression ( -z9 ). I'm 
not sure if the Java implementation in the SDK supports this level of 
compression. It may be that we have to switch to a native implementation 
distributed with the SDK instead.

Q: Is there a License issue with the LZMA SDK code?

All the code in the latest LZMA 9.20 SDK is now completely in the public 
domain so I don't foresee any license issues. The LZMA SDK Java 
implementation is used in the Grizzly HTTP Web Framework, Apache Ant, 
and at least two other Java LZMA binding projects. Native tools like 
7zip and lzma (on Linux) all appear to be using the native LZMA SDK 
implementation.

Q: What next?

I would like to spend some more time trying the native LZMA 
implementation. I think we can get close to the debian package size with 
it. I also need to spend some more time analyzing the decompression cost.

I believe LZMA would be a good candidate for a public API in OpenJDK, 
say java.util.[zip|jar].LZMA. I've done a rough prototype, very similar 
to Pack200 ( static factory methods for creating compressor/decompressor 
). I envisage that the OpenJDK LZMA implementation would simply leverage 
the LZMA SDK implementation.

-Chris.

[1] http://www.7-zip.org/sdk.html