Improving AppCDS for Custom Loaders

Jiangli Zhou jiangli.zhou at oracle.com
Fri May 4 17:42:41 UTC 2018 

Hi Volker,

Thank you so much for the feedbacks!

You comments made me realize that some clarifications are need for the top-layer dynamic archiving. The top-layer would include all application classes, middleware classes, system library classes that are not in the bottom layer. Apologizing for not stating that clearly in the proposal. With the top layer including all necessary classes (excluding the classes in bottom layer) for a specific application, multiple instances running the same application can achieve maximum memory sharing. Using your example, the extra AWT/Swing classes are archived along with the application classes in the top layer.

With the dynamic archiving ability, it’s natural to think there is no need for the two-layer scheme. In fact, I originally had the thoughts. However, after exploring different possible use cases, I agree two-layer archiving do bring benefits that are lacking in the single archive design. One example of such use case is tens/hundreds different applications running on the same machine. Without a common system layer archive, it is difficult to achieve memory sharing for common system classes, String objects, etc between different applications. 

Another reason why we should go with a two-layer & static/dynamic archiving is some runtime information may be sensitive and should not be archived. Strings can be a good example in this case. Currently, we archive interned String objects without running application. The Strings include resolved CONSTANT_strings, class names, method names, etc, which are all static information from class files. During execution of an application, interned Strings might contain sensitive user information, which is not safe to be included in the archive. It’s difficult to distinguish between the static information and runtime information if we go with the single layer dynamic-only archiving.

We had many discussions internally over long period of time on how to improve usability with different approaches. In the end, dynamic-only or static-only two-layer archiving all have their own disadvantages and fail to meet requirements in some use cases. The hybrid archiving combines benefits/advantages of different approaches and seem to be flexible enough to fit most usages.

Further comments and feedbacks are much appreciated.

Best regards,

Jiangli

> On May 4, 2018, at 3:01 AM, Volker Simonis <volker.simonis at gmail.com> wrote:
> 
> Hi Jiangli,
> 
> thanks for sharing the hybrid archiving proposal. I think it is very
> interesting and useful!
> 
> One issue I see is that the "system library classes" which should go
> into the "static" archive are inherently application specific (i.e. if
> my application uses Swing, it will use most of the AWT/Swing classes).
> So how will you cope with this problem. Either you put ALL the system
> classes into the static archive which will be a waste for most
> applications. Or you just put a small commonly used subset of classes
> into the static archive which may miss many classes for specific
> applications.
> 
> If we would add the possibility to create a dynamic archive at runtime
> / program end (which I think would be great from a usability
> perspective) I don't see a big need for two different archives. Two
> archives will further complicate (and slow down) things like Symbol
> lookup (we already have two SymbolTable now and we'd probably need a
> third one if we would have two archives).
> 
> I don't think that running a few different Java applications on one
> host is the most important use case for CDS. In such a scenario the
> current, build time generated archive is probably the best we can do
> anyway. Instead, I think the most important use case is if we have
> many instances of the same Java application running on the same host.
> And this is becoming more common with the raise of containerization.
> For the latter use case, a dynamically generated, application specific
> archive would be the optimal solution.
> 
> Thank you and best regards,
> Volker
> 
> 
> 
> On Fri, May 4, 2018 at 3:42 AM, Jiangli Zhou <jiangli.zhou at oracle.com> wrote:
>> Hi Volker,
>> 
>> Here are some details about the hybrid archiving. The goal is to harvest the benefit of archiving by default and improve its usability. The hybrid approach combines a two-layer archiving (proposed by Ioi internally) and static & dynamic archiving techniques:
>> 
>> - Statically archive system library classes from a provided classlist using the existing method. The archiving includes class metadata, interned string objects, constant pool resolved_references arrays, class mirror objects, etc. Static archiving can be done at the JDK image build time and shipped together with JDK binary. Following need to be addressed:
>>        *Relaxing the runtime CDS/AppCDS boot path check, so the packaged archive can be used after the JDK binary is installed on the target device. JDK-8199807 was created to address this issue and is targeted for JDK 11.
>>        *Add the static archiving generation in JDK build steps and package the generated archive with JDK image. The archive can only be generated for the same target (both OS can CPU architecture) as the build platform. I will create a RFE.
>> 
>> - Dynamic archiving can done for application classes at the first execution of a specific application
>>        * The archive is created on top of the default system archive shipped with the JDK image. A separate top-layer archive file is generated for each different application.
>>        * Archiving is done at the end of the application execution before VM exists by relocating the class metadata to the archive spaces. Cleanup also needs to be done for copied class meta data to remove any runtime information. Most of the required functionality already exists today. For example, class metadata relocation was implemented by Ioi in JDK 10.
>>        * Only archive class metadata for application in the top layer initially. Archiving java heap objects in the top-layer requires more investigations.
>> 
>> Benefits of the hybrid archiving:
>> * The system archive can be shared by different applications and provides memory saving.
>> * Archive for application is created and used transparently. No more profiling step and class list are required!
>> * Separating the system archiving from application archiving reduces the cost of archiving at application execution time. The overhead added to the first execution time is reduced.
>> 
>> Thanks,
>> 
>> Jiangli
>> 
>> 
>>> On May 3, 2018, at 10:34 AM, Jiangli Zhou <jiangli.zhou at Oracle.COM> wrote:
>>> 
>>> 
>>>> On May 3, 2018, at 2:14 AM, Volker Simonis <volker.simonis at gmail.com> wrote:
>>>> 
>>>> On Thu, May 3, 2018 at 11:01 AM, David Holmes <david.holmes at oracle.com> wrote:
>>>>> On 3/05/2018 5:16 PM, Volker Simonis wrote:
>>>>>> 
>>>>>> On Thu, May 3, 2018 at 8:55 AM, David Holmes <david.holmes at oracle.com>
>>>>>> wrote:
>>>>>>> 
>>>>>>> Just lurking here but ...
>>>>>>> 
>>>>>>>> But is this really y relevant use case? Why would I like to create ONE
>>>>>>>> archive for several apps? This would actually increase the footprint
>>>>>>>> of a single instance which uses this archive. If I have several apps I
>>>>>>>> would expect that users create a specific archive for each app to get
>>>>>>>> the best out of CDS.
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> One app instance may get increased footprint but you presumably use CDS
>>>>>>> because you have multiple apps running (whether the same or not). These
>>>>>>> apps
>>>>>>> all share the core JDK classes from the archive so the overall footprint
>>>>>>> per
>>>>>>> instance is less.
>>>>>>> 
>>>>>> 
>>>>>> If we just want to share the core JDK classes that's easy. For that we
>>>>>> could mostly use the default class list (or a slightly extended one)
>>>>>> which is generated at JDK build time (at JAVA_HOME/lib/classlist).
>>>>> 
>>>>> 
>>>>> The point is that you are presumably running multiple instances of multiple
>>>>> apps, hence you want to share one set of core classes across all, and share
>>>>> the app classes across each app instance.
>>>>> 
>>>> 
>>>> But that would require two archives: a general one with the core
>>>> classes and an application specific one for each application.
>>>> Combining the core classes and the application of various applications
>>>> will not be optimal because the application classes will be all mixed
>>>> in the same archive. The archive is being mapped page-wise into the
>>>> java process so you'll probably end up mapping the whole archive into
>>>> each process although you'll only use a fraction of the classes in the
>>>> archive.
>>>> 
>>>>>> If we want to use ONE archive for several applications and we can
>>>>>> accept to have a bigger footprint if running a single (or just a few)
>>>>>> applications in parallel I suppose the overhead of simply dumping all
>>>>>> the classes from the classpathes of the various applications compared
>>>>>> to an accurate solution where we only dump the actually used classes
>>>>>> of all applications would be not that big.
>>>>> 
>>>>> 
>>>>> But those "accurate" solutions duplicate the core classes and that's a waste
>>>>> of footprint.
>>>>> 
>>>> 
>>>> By "accurate" I meant one "fat" archive which contains all the classes
>>>> USED by several applications plus the core classes. My argument was
>>>> that such an "accurate" "fat" archive won't be much smaller compared
>>>> to a "fat" archive which simply contains all the core classes plus all
>>>> the application classes (i.e. from the application class pathes, no
>>>> matter if they are ever used or not). But the latter would be much
>>>> simpler to implement.
>>> 
>>> The above discussion and an internal proposal for hybrid archiving seem to converge on a few points. If there is no objection to the hybrid archiving proposal internally, maybe we can shared the details of the proposal on openjdk soon.
>>> 
>>> Thanks,
>>> 
>>> Jiangli
>>> 
>>> 
>>>> 
>>>>> David
>>>>> -----
>>>>> 
>>>>> 
>>>>>>> David
>>>>>>> -----
>>>>>>> 
>>>>>>> 
>>>>>>> On 3/05/2018 4:48 PM, Volker Simonis wrote:
>>>>>>>> 
>>>>>>>> 
>>>>>>>> On Thu, May 3, 2018 at 6:52 AM, Ioi Lam <ioi.lam at oracle.com> wrote:
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> On 5/2/18 10:00 AM, Volker Simonis wrote:
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> On Tue, May 1, 2018 at 8:32 PM, Ioi Lam <ioi.lam at oracle.com> wrote:
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> PROBLEM:
>>>>>>>>>>> 
>>>>>>>>>>> As discussed with Volker and Yumin in previous e-mails, AppCDS has
>>>>>>>>>>> some
>>>>>>>>>>> experimental support for custom class loaders. However, it's not very
>>>>>>>>>>> easy
>>>>>>>>>>> to use.
>>>>>>>>>>> 
>>>>>>>>>>> For example, you can write a classlist like this:
>>>>>>>>>>> 
>>>>>>>>>>>     java/lang/Object id: 1
>>>>>>>>>>>     CustomLoadee id: 2 super: 1 source: /tmp/foo.jar
>>>>>>>>>>> 
>>>>>>>>>>> The CustomLoadee class will be stored in the shared archive with a
>>>>>>>>>>> CRC
>>>>>>>>>>> code.
>>>>>>>>>>> During runtime, if a customed loader wants to load a class of the
>>>>>>>>>>> same
>>>>>>>>>>> name,
>>>>>>>>>>> and its classfile has the same size and CRC as the archived class,
>>>>>>>>>>> the
>>>>>>>>>>> archived version will be loaded. This speeds up class loading by
>>>>>>>>>>> avoiding
>>>>>>>>>>> parsing the class file, and saves space by sharing the mmap'ed class
>>>>>>>>>>> metadata across processes.
>>>>>>>>>>> 
>>>>>>>>>>> You can see an example test at:
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> http://hg.openjdk.java.net/jdk/hs/file/46dc568d6804/test/hotspot/jtreg/runtime/appcds/customLoader/HelloCustom.java
>>>>>>>>>>> 
>>>>>>>>>>> However, the current scheme requires you to specify all the super
>>>>>>>>>>> classes
>>>>>>>>>>> and interfaces. There's no support provided by the
>>>>>>>>>>> -XX:DumpLoadedClassList
>>>>>>>>>>> option. It can be helped somewhat with Volker's tool:
>>>>>>>>>>> https://github.com/simonis/cl4cds
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> POSSIBLE SOLUTIONS:
>>>>>>>>>>> 
>>>>>>>>>>> 1. "Dump-as-you-go". As suggested by Yumin, we can provide a jcmd to
>>>>>>>>>>> ask
>>>>>>>>>>> a
>>>>>>>>>>> running JVM process to dump all of its loaded classes, including
>>>>>>>>>>> those
>>>>>>>>>>> loaded by custom loaders, into an archive. An alternative is to dump
>>>>>>>>>>> the
>>>>>>>>>>> archive at JVM exit time (or when you press Ctrl-C, etc.
>>>>>>>>>>> 
>>>>>>>>>>> 2. Add information about the custom classes for
>>>>>>>>>>> -XX:DumpLoadedClassList.
>>>>>>>>>>> The
>>>>>>>>>>> trouble is some class loaders don't specify a code source that can be
>>>>>>>>>>> understood by the built-in class loaders. For example, the "Fat Jars"
>>>>>>>>>>> would
>>>>>>>>>>> have a code source like
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> jar:file:/jdk/tmp/test-1.0-SNAPSHOT.jar!/BOOT-INF/lib/validation-api-2.0.1.Final.jar!/
>>>>>>>>>>> 
>>>>>>>>>>> also, many custom loaders would pre-process the classfile data before
>>>>>>>>>>> defining the class, so we can't simply archive the version of the
>>>>>>>>>>> class
>>>>>>>>>>> on
>>>>>>>>>>> disk.
>>>>>>>>>>> 
>>>>>>>>>>> One possible solution for #2 is to include the class file data in the
>>>>>>>>>>> -XX:DumpLoadedClassList output:
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>>     java/lang/Object id: 1
>>>>>>>>>>>     CustomLoadee id: 2 super: 1 source: base64
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> yv66vgAAADQAFwoABAAQCQAFABEHABIHABMHABQBAAJIaQEADElubmVyQ2xhc3NlcwEABjxpbml0
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> PgEAAygpVgEABENvZGUBAA9MaW5lTnVtYmVyVGFibGUBAAFmAQADKClJAQAKU291cmNlRmlsZQEA
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> CkhlbGxvLmphdmEMAAgACQwAFQAWAQAFSGVsbG8BABBqYXZhL2xhbmcvT2JqZWN0AQAISGVsbG8k
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> SGkBAAFYAQABSQAhAAMABAAAAAAAAgABAAgACQABAAoAAAAdAAEAAQAAAAUqtwABsQAAAAEACwAA
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> AAYAAQAAAC8ACAAMAA0AAQAKAAAAHAABAAAAAAAEsgACrAAAAAEACwAAAAYAAQAAADEAAgAOAAAA
>>>>>>>>>>>     AgAPAAcAAAAKAAEABQADAAYACA==
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> Of the 2 solutions:
>>>>>>>>>>> 
>>>>>>>>>>> #1 seems easier to use, but may require more invasive modifications
>>>>>>>>>>> in
>>>>>>>>>>> the
>>>>>>>>>>> VM, especially if you want to be able to continue execution after
>>>>>>>>>>> dumping.
>>>>>>>>>>> 
>>>>>>>>>> Not sure what #1 really proposes: dumping the complete .jsa archive at
>>>>>>>>>> runtime or dumping just the loaded classes.
>>>>>>>>>> 
>>>>>>>>>> If it's just about dumping the loaded class without generating the
>>>>>>>>>> .jsa archive there's the problem that by default the VM doesn't store
>>>>>>>>>> the exact bytes of a class after the class was loaded (except when
>>>>>>>>>> class transformers are registered). So the class files would have to
>>>>>>>>>> be re-assembled from the internal VM structures (in the same way this
>>>>>>>>>> is done for class redefinition) and the resulting class-file may be
>>>>>>>>>> different from the original bytes (i.e. some attributes may be
>>>>>>>>>> missing).
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> #1 is for creating the JSA file, not just dumping the class files.
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> If #1 is about creating the whole .jsa archive at runtime (or at VM
>>>>>>>>>> exit) I think that would be the most attractive solution from a
>>>>>>>>>> usability point of view although I understand that #2 will be easier
>>>>>>>>>> to implement in the short term. Regarding the argument that #1 will
>>>>>>>>>> produce a "binary blob" that's true, but that's already true now when
>>>>>>>>>> we use "Xshare:dump". I think it should be not to hard to implement a
>>>>>>>>>> tool based an SA which could introspect a .jsa archive.
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> The argument about the binary blob is to compare it against the text
>>>>>>>>> file
>>>>>>>>> produced by -XX:DumpLoadedClassList.
>>>>>>>>> 
>>>>>>>>> One use case to consider is when you have a JAR file that contains
>>>>>>>>> several
>>>>>>>>> apps that each load a unique set of classes. Today, (assuming that
>>>>>>>>> custom
>>>>>>>>> class loaders are not used), you can run each app once with
>>>>>>>>> -XX:DumpLoadedClassList, and then do an
>>>>>>>>> 
>>>>>>>>>    cat *.classlist | sort | uniq > combined.classlist
>>>>>>>>> 
>>>>>>>>> and then create an archive that would work for all these apps.
>>>>>>>>> 
>>>>>>>> 
>>>>>>>> But is this really y relevant use case? Why would I like to create ONE
>>>>>>>> archive for several apps? This would actually increase the footprint
>>>>>>>> of a single instance which uses this archive. If I have several apps I
>>>>>>>> would expect that users create a specific archive for each app to get
>>>>>>>> the best out of CDS.
>>>>>>>> 
>>>>>>>>> With the binary blob, there's no easy way of doing this. It will be
>>>>>>>>> very
>>>>>>>>> difficult to write a tool to decipher each blob and then somehow
>>>>>>>>> combine
>>>>>>>>> them into a single one.
>>>>>>>>> 
>>>>>>>> 
>>>>>>>> But if users really wants such a "fat" archive, there's a much easier
>>>>>>>> way: just dump ALL the classes from the .jar file into the archive. A
>>>>>>>> class list for this could easily be assembled either with an external
>>>>>>>> tool like cl4cds (or even a simple shell scripts which converts the
>>>>>>>> output of `unzip -l <jar-file>` into the correct format). Or, even
>>>>>>>> simpler, by adding a new option to the VM similar to
>>>>>>>> -XX:DumpLoadedClassList which dumps all the classes it can find on the
>>>>>>>> class path (and potentially other, configurable locations).
>>>>>>>> 
>>>>>>>>> Thanks
>>>>>>>>> - Ioi
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>>> #2 would be easier to implement, but the classlist might be huge.
>>>>>>>>>>> 
>>>>>>>>>>> Also, #2 would allow post-processing tools to remove unneeded
>>>>>>>>>>> classes,
>>>>>>>>>>> or
>>>>>>>>>>> merge two runs into a single list. The output of #1 is essentially a
>>>>>>>>>>> binary
>>>>>>>>>>> blob that's impossible for off-line analysis/optimizations.
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> Any comments, or suggestions for alternatives?
>>>>>>>>>>> 
>>>>>>>>>>> Thanks
>>>>>>>>>>> - Ioi
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>> 
>>>>>>> 
>>>>> 
>>> 
>>

More information about the hotspot-runtime-dev mailing list