Improving AppCDS for Custom Loaders

[Jiangli Zhou]

Hi Yumin,

Thanks for the feedback and questions.


On 12/19/18 2:04 PM, yumin qi wrote:
> HI, Jiangli
>
>   Thanks for implementing the dynamic dump feature.
>   I have some questions:
>   1) Base archive. Is it only including boot images (java.base, 
> rt.jar, java.net <http://java.net> ... )? Does it include ext classes?

The base layer is the default CDS archive packaged with a JDK binary 
(starting from JDK 12). The default CDS archive contains classes 
specified in the JDK ./lib/classlist, which is also automatically 
generated at JDK build time. All the archived classes are from the 
runtime image and vast majority of those classes are the ones used 
during JVM bootstraping.

If you want to support dynamic archiving for JDK 8 and include extension 
classes in your base layer, you can add them to your classlist when 
creating the base archive.

>   2) The dump happens at the exit of application. There is maybe a 
> case that at exit, many classes already unloaded. If they are not in 
> shared archive and have big number, it will affect startup performance 
> since they will be loaded from source.

We are currently working on two approaches in parallel:

1) Always keep custom class loaders and their loaded classes alive
2) Do extra buffering of the metadata for the classes from custom 
loaders before unloading

With both approaches, the metadata of the classes for custom loaders can 
be archived without being missed. The approach 1) has been done (thanks 
Calvin for investigating) already and is used for our testing today.

>        So maybe, as you described in bug, copy the custom loader 
> loaded classes to shared space when they are first time loaded.

With the approach 2), we can do the extra buffering either during the 
creation time or unloading time (of a specific custom class loader).

>   3) What you refer to custom classes, do they include classes which 
> are loaded by app loader loaded classes?

In the top layer archive, all dynamically loaded (not in the base 
archive) classes are archived (except some excluded classes). Those 
classes include the ones dynamically loaded by the boot loader, 
PlatformClassLoader, system loader (the AppClassLoader) and the user 
defined class loaders (custom class loaders). Hope it clarifies.

>   4) Does base archive take more space than (current) using class list 
> file? Not every class in base archive is used for an application, so I 
> support the other choice in your work ---- archive one for all (both 
> base and top layer).

The base archive is the default CDS archive packaged with JDK, which 
contains only essential classes that are commonly used by Java 
applications and is generated from a classlist (please see above as 
well). The default archive provides the best balance between JVM startup 
time performance, runtime and static footprint (thanks for the work and 
help from our performance team, especially Claes).

User can recreate the base archive if they do want to add or remove any 
classes.

>   5) This should not be a question. dynamicDump, Can we give a short 
> name for command line option. No one (like me) like typing a long name 
> with Capital in it. Maybe ddump or dydump is enough?

Do you have any suggestion on the option name?

Thanks!
Jiangli

> --Yumin
>
> On Mon, Dec 17, 2018 at 5:17 PM Jiangli Zhou <jiangli.zhou at oracle.com 
> <mailto:jiangli.zhou at oracle.com>> wrote:
>
>     Hi Volker, Yumin and others,
>
>     This is a follow up of our email discussions (back in May of this
>     year)
>     on improving AppCDS for custom class loaders. An umbrella RFE,
>     JDK-8202854 (Hybrid archiving,
>     https://bugs.openjdk.java.net/browse/JDK-8202854) was created in
>     May to
>     track the activities related to the topic. We have now started
>     working
>     on the final subtask
>     (https://bugs.openjdk.java.net/browse/JDK-8207812)
>     to implement dynamic archiving. There are some updates added to
>     JDK-8207812. If you have any further feedback, please share with us
>     (thanks!).
>
>     With dynamic archiving, we can support custom class loaders more
>     effectively. A class list is no longer needed when creating an AppCDS
>     archive dynamically (during a java application execution). That
>     gives us
>     a potential to cleanup our code base and remove the experimental
>     support
>     for the custom loader's class list (mainly classListParser.cpp and
>     classListParser.hpp). The cleanup can be staged after dynamic
>     archiving
>     is matured. Please let us know if you see any issues with the plan.
>
>     Thanks!
>
>     Jiangli
>