RFR: 8150044: Generate classlists at build-time

[Claes Redestad]

Hi Ioi,

On 2016-05-04 16:05, Ioi Lam wrote:
> Hi Claes,
>
> The HotSpot changes look good to me.

thanks!

>
> I am not knowledgeable enough to comment on the top repo makefile 
> changes.
>
> Regarding the JDK changes, in HelloClasslist.java:
>
> Maybe add a comment about how/why you choose this particular set of 
> operations? When the JDK evolves in the future, how should this file 
> be changed?

I've done a few cycles of experiments to get something that shows a good 
improvement on our minimal and server-style startup test, while 
mitigating the regressions we'll inevitably see
on some of the GUI apps.

It'd definitely be interesting to iterate a bit on this as the JDK 
evolves, and I think one of the benefits of this solution is that such 
experiments are quite straightforward to setup.

>
> I notice that no GUI classes are used. Is the reason (a) GUI classes 
> are not important anymore, or (b) the build would fail in a headless 
> environment if GUI classes were used by HelloClasslist.java?

A bit of both:

(b) Yes, requiring build environments to have a graphical environment is 
complicated, but not impossible. We'd prefer not to have add that 
requirement.

(a) Yes and no... while GUI classes aren't unimportant in and off 
themselves, jigsaw, java.lang.invoke and other features are adding to 
the number of java classes needed to bootstrap Java, and thus we'd see 
the CDS archive grow ever larger if we simply kept refreshing the 
classlists using the same technique as before. This actually has other 
drawbacks such as reducing the chance the CDS archive will be 
successfully mapped, especially on 32-bit platforms with ASLR. For 
example on Windows 32-bit startup tests, the regression on GUI apps is 
mostly offset by having a higher success rate.

Thanks!

/Claes

>
> Thanks
> - Ioi
>
>
> On 5/4/16 6:36 AM, Claes Redestad wrote:
>> Hi,
>>
>> please review this change to generate classlists at build-time
>>
>> bug: https://bugs.openjdk.java.net/browse/JDK-8150044
>>
>> webrevs:
>> top: http://cr.openjdk.java.net/~redestad/8150044/top.01/
>> jdk: http://cr.openjdk.java.net/~redestad/8150044/jdk.01/
>> hotspot: http://cr.openjdk.java.net/~redestad/8150044/hotspot.01/
>>
>> The implementation generates an interim image consisting of a minimal
>> set of modules, then use this to run a small generator program to
>> load common utilities and facilities and dump the result of this to a
>> classlist that is then bundled with the final images.
>>
>> The smaller number of classes on the default classlist (~1100 instead
>> of ~2500) requires some adjustment to the metaspace defaults.
>>
>> This achieves the following:
>>
>> - Removes a manual, error-prone process to update the versioned
>>   classlists
>> - Ensures the classlists shipped with the JDK/JRE is up to date
>>   with recent JDK changes, e.g., when moving classes from sun.* to
>>   jdk.internal.*
>> - Automatically picks up and incorporates the output of jlink plugins
>>   such as GenerateJLIClassesPlugin into the classlist
>> - Supports cross-compilation build targets, although it runs using a
>>   build JDK that can run on the host platform to generate such
>>   classlists (this isn't ideal, but no worse than the current
>>   situation, where the versioned classlist for the host platform is
>>   simply copied to the cross-compiled target)
>>
>> There are a few concerns/drawbacks:
>>
>> - It does add complexity to the build, and concern has been voiced that
>>   this would adversely affect build times. However, I'm happy to say
>>   that on my machine build times are roughly the same:
>>
>> Before:
>> real    2m37.303s
>> user    35m33.576s
>> sys    3m46.476s
>>
>> After:
>> real    2m36.168s
>> user    35m31.232s
>> sys    3m52.268s
>>
>> (real time varies ± 5s from build to build)
>>
>> - Startup on the specific applications we've used to generate the
>>   classlists for previously suffer small regressions. These are
>>   specifically rather dated AWT and Swing-based applications. OTOH,
>>   startup characteristics generally improve on other applications
>>   (minimal VM, jetty, etc...)
>>
>> Testing: JPRT -testset hotspot
>>
>> Thanks!
>>
>> /Claes
>