Improving AppCDS for Custom Loaders

[Ioi Lam]

On 5/10/18 4:05 PM, Ioi Lam wrote:
>
>
> On 5/10/18 3:34 PM, Volker Simonis wrote:
>>
>> yumin qi <yumin.qi at gmail.com <mailto:yumin.qi at gmail.com>> schrieb am 
>> Do. 10. Mai 2018 um 22:20:
>>
>>     Hi, Jiangli
>>
>>       Thanks for the info.
>>
>>     On Thu, May 10, 2018 at 11:39 AM, Jiangli Zhou
>>     <jiangli.zhou at oracle.com <mailto:jiangli.zhou at oracle.com>> wrote:
>>
>>         Hi Yumin and Volker,
>>
>>         I’ve create an umbrella RFE JDK-8202854 to keep track this
>>         effort. New sub-RFEs will be created and targeted for specific
>>         JDK releases when each sub-item matures. Please refer to
>>         JDK-8202854 for future progress.
>>
>>         Yumin, please see below for comments/questions.
>>
>>         > Agree not including objects in java heap at now --- thanks 
>> Jiangli for answering my question
>>         in other thread.
>>
>>         I conclude from above that you are not using java heap object
>>         archiving due to different GC algorithm being used(?). I
>>         would like to understand more about your GC choices. Could you
>>         please provide more information on that? G1 GC is the default
>>         GC in JDK. Thanks to our GC team, the performance of G1 has
>>         been consistently improving over the recent JDK releases,
>>         especially in JDK 9 and 10. I strongly encourage you to do
>>         performance measurement/comparison and reevaluate your GC
>>         choice if G1 is not used.
>>
>>
>>     Most of our applications still using CMS, we are pushing to switch
>>     to G1GC at this time but still most of them still with CMS. I know
>>     CMS will be removed so it will not be your focus.
>>
>>
>>         >   I think for #1, it does not conflict with the two layer
>>         archive solution. We can skip the classes from base CDS
>>         archive, only dump the non-base loaded classes into second
>>         archive, this gives one more option for user to choose. Also
>>         it will be good after dump archive to let the application
>>         continue to run.
>>         >
>>         >   Can we do a Full GC before dump at exit? it may unload not
>>         referenced classes, or it is not necessary since CDS wants to
>>         resolve the startup performance for all loaded classes?
>>
>>         For classes loaded by the builtin loaders, they can be kept
>>         alive when archiving phase start. For classes loaded by user
>>         defined class loaders, we also need to prevent them from being
>>         unloaded before archiving. I have a preliminary idea for how
>>         to do that. I can share more information on that when it 
>> matures.
>>
>>
>>     Yes, if classes with custom class loader cleaned, the class loader
>>     itself will be purged. Full GC before exit may not be a good idea,
>>     but we can skip unloading classes --- in case dump objects in java
>>     heap.
>>
>>
>> Why not dump the classes iteratively, while they are loaded. Class 
>> loading time would be the most natural time for dumping a class, and 
>> it‘s the time when we know the most about a class (i.e. full class 
>> bytes).
>>
>
> The archive is divided into RO/RW sections. Mutable metadata such as 
> InstanceKlasses are stored in RW, while non-mutable ones such as 
> ConstMethod and Symbol are stored into the RO section. In order to 
> have a single section each for RW and RO, currently we load all the 
> classes first, and then calculate the sizes of the 2 sections.
>
>
One possible solution is to copy the loaded classes into a temp memory 
buffer, as the classes are being loaded. Then, when we decide to dump 
the archive, we can iterate over all the classes in the temp memory buffer.

- Ioi


>>     Thanks
>>     Yumin
>>
>>
>