Improving AppCDS for Custom Loaders

[Ioi Lam]

On 5/2/18 10:00 AM, Volker Simonis wrote:
> On Tue, May 1, 2018 at 8:32 PM, Ioi Lam <ioi.lam at oracle.com> wrote:
>> PROBLEM:
>>
>> As discussed with Volker and Yumin in previous e-mails, AppCDS has some
>> experimental support for custom class loaders. However, it's not very easy
>> to use.
>>
>> For example, you can write a classlist like this:
>>
>>      java/lang/Object id: 1
>>      CustomLoadee id: 2 super: 1 source: /tmp/foo.jar
>>
>> The CustomLoadee class will be stored in the shared archive with a CRC code.
>> During runtime, if a customed loader wants to load a class of the same name,
>> and its classfile has the same size and CRC as the archived class, the
>> archived version will be loaded. This speeds up class loading by avoiding
>> parsing the class file, and saves space by sharing the mmap'ed class
>> metadata across processes.
>>
>> You can see an example test at:
>>
>> http://hg.openjdk.java.net/jdk/hs/file/46dc568d6804/test/hotspot/jtreg/runtime/appcds/customLoader/HelloCustom.java
>>
>> However, the current scheme requires you to specify all the super classes
>> and interfaces. There's no support provided by the -XX:DumpLoadedClassList
>> option. It can be helped somewhat with Volker's tool:
>> https://github.com/simonis/cl4cds
>>
>>
>> POSSIBLE SOLUTIONS:
>>
>> 1. "Dump-as-you-go". As suggested by Yumin, we can provide a jcmd to ask a
>> running JVM process to dump all of its loaded classes, including those
>> loaded by custom loaders, into an archive. An alternative is to dump the
>> archive at JVM exit time (or when you press Ctrl-C, etc.
>>
>> 2. Add information about the custom classes for -XX:DumpLoadedClassList. The
>> trouble is some class loaders don't specify a code source that can be
>> understood by the built-in class loaders. For example, the "Fat Jars" would
>> have a code source like
>>
>>
>> jar:file:/jdk/tmp/test-1.0-SNAPSHOT.jar!/BOOT-INF/lib/validation-api-2.0.1.Final.jar!/
>>
>> also, many custom loaders would pre-process the classfile data before
>> defining the class, so we can't simply archive the version of the class on
>> disk.
>>
>> One possible solution for #2 is to include the class file data in the
>> -XX:DumpLoadedClassList output:
>>
>>
>>      java/lang/Object id: 1
>>      CustomLoadee id: 2 super: 1 source: base64
>>
>> yv66vgAAADQAFwoABAAQCQAFABEHABIHABMHABQBAAJIaQEADElubmVyQ2xhc3NlcwEABjxpbml0
>>
>> PgEAAygpVgEABENvZGUBAA9MaW5lTnVtYmVyVGFibGUBAAFmAQADKClJAQAKU291cmNlRmlsZQEA
>>
>> CkhlbGxvLmphdmEMAAgACQwAFQAWAQAFSGVsbG8BABBqYXZhL2xhbmcvT2JqZWN0AQAISGVsbG8k
>>
>> SGkBAAFYAQABSQAhAAMABAAAAAAAAgABAAgACQABAAoAAAAdAAEAAQAAAAUqtwABsQAAAAEACwAA
>>
>> AAYAAQAAAC8ACAAMAA0AAQAKAAAAHAABAAAAAAAEsgACrAAAAAEACwAAAAYAAQAAADEAAgAOAAAA
>>      AgAPAAcAAAAKAAEABQADAAYACA==
>>
>>
>> Of the 2 solutions:
>>
>> #1 seems easier to use, but may require more invasive modifications in the
>> VM, especially if you want to be able to continue execution after dumping.
>>
> Not sure what #1 really proposes: dumping the complete .jsa archive at
> runtime or dumping just the loaded classes.
>
> If it's just about dumping the loaded class without generating the
> .jsa archive there's the problem that by default the VM doesn't store
> the exact bytes of a class after the class was loaded (except when
> class transformers are registered). So the class files would have to
> be re-assembled from the internal VM structures (in the same way this
> is done for class redefinition) and the resulting class-file may be
> different from the original bytes (i.e. some attributes may be
> missing).
#1 is for creating the JSA file, not just dumping the class files.
> If #1 is about creating the whole .jsa archive at runtime (or at VM
> exit) I think that would be the most attractive solution from a
> usability point of view although I understand that #2 will be easier
> to implement in the short term. Regarding the argument that #1 will
> produce a "binary blob" that's true, but that's already true now when
> we use "Xshare:dump". I think it should be not to hard to implement a
> tool based an SA which could introspect a .jsa archive.
The argument about the binary blob is to compare it against the text 
file produced by -XX:DumpLoadedClassList.

One use case to consider is when you have a JAR file that contains 
several apps that each load a unique set of classes. Today, (assuming 
that custom class loaders are not used), you can run each app once with 
-XX:DumpLoadedClassList, and then do an

     cat *.classlist | sort | uniq > combined.classlist

and then create an archive that would work for all these apps.

With the binary blob, there's no easy way of doing this. It will be very 
difficult to write a tool to decipher each blob and then somehow combine 
them into a single one.

Thanks
- Ioi

>> #2 would be easier to implement, but the classlist might be huge.
>>
>> Also, #2 would allow post-processing tools to remove unneeded classes, or
>> merge two runs into a single list. The output of #1 is essentially a binary
>> blob that's impossible for off-line analysis/optimizations.
>>
>>
>> Any comments, or suggestions for alternatives?
>>
>> Thanks
>> - Ioi
>>
>>