Improving AppCDS for Custom Loaders

[Volker Simonis]

On Thu, May 3, 2018 at 6:52 AM, Ioi Lam <ioi.lam at oracle.com> wrote:
>
>
> On 5/2/18 10:00 AM, Volker Simonis wrote:
>>
>> On Tue, May 1, 2018 at 8:32 PM, Ioi Lam <ioi.lam at oracle.com> wrote:
>>>
>>> PROBLEM:
>>>
>>> As discussed with Volker and Yumin in previous e-mails, AppCDS has some
>>> experimental support for custom class loaders. However, it's not very
>>> easy
>>> to use.
>>>
>>> For example, you can write a classlist like this:
>>>
>>>      java/lang/Object id: 1
>>>      CustomLoadee id: 2 super: 1 source: /tmp/foo.jar
>>>
>>> The CustomLoadee class will be stored in the shared archive with a CRC
>>> code.
>>> During runtime, if a customed loader wants to load a class of the same
>>> name,
>>> and its classfile has the same size and CRC as the archived class, the
>>> archived version will be loaded. This speeds up class loading by avoiding
>>> parsing the class file, and saves space by sharing the mmap'ed class
>>> metadata across processes.
>>>
>>> You can see an example test at:
>>>
>>>
>>> http://hg.openjdk.java.net/jdk/hs/file/46dc568d6804/test/hotspot/jtreg/runtime/appcds/customLoader/HelloCustom.java
>>>
>>> However, the current scheme requires you to specify all the super classes
>>> and interfaces. There's no support provided by the
>>> -XX:DumpLoadedClassList
>>> option. It can be helped somewhat with Volker's tool:
>>> https://github.com/simonis/cl4cds
>>>
>>>
>>> POSSIBLE SOLUTIONS:
>>>
>>> 1. "Dump-as-you-go". As suggested by Yumin, we can provide a jcmd to ask
>>> a
>>> running JVM process to dump all of its loaded classes, including those
>>> loaded by custom loaders, into an archive. An alternative is to dump the
>>> archive at JVM exit time (or when you press Ctrl-C, etc.
>>>
>>> 2. Add information about the custom classes for -XX:DumpLoadedClassList.
>>> The
>>> trouble is some class loaders don't specify a code source that can be
>>> understood by the built-in class loaders. For example, the "Fat Jars"
>>> would
>>> have a code source like
>>>
>>>
>>>
>>> jar:file:/jdk/tmp/test-1.0-SNAPSHOT.jar!/BOOT-INF/lib/validation-api-2.0.1.Final.jar!/
>>>
>>> also, many custom loaders would pre-process the classfile data before
>>> defining the class, so we can't simply archive the version of the class
>>> on
>>> disk.
>>>
>>> One possible solution for #2 is to include the class file data in the
>>> -XX:DumpLoadedClassList output:
>>>
>>>
>>>      java/lang/Object id: 1
>>>      CustomLoadee id: 2 super: 1 source: base64
>>>
>>>
>>> yv66vgAAADQAFwoABAAQCQAFABEHABIHABMHABQBAAJIaQEADElubmVyQ2xhc3NlcwEABjxpbml0
>>>
>>>
>>> PgEAAygpVgEABENvZGUBAA9MaW5lTnVtYmVyVGFibGUBAAFmAQADKClJAQAKU291cmNlRmlsZQEA
>>>
>>>
>>> CkhlbGxvLmphdmEMAAgACQwAFQAWAQAFSGVsbG8BABBqYXZhL2xhbmcvT2JqZWN0AQAISGVsbG8k
>>>
>>>
>>> SGkBAAFYAQABSQAhAAMABAAAAAAAAgABAAgACQABAAoAAAAdAAEAAQAAAAUqtwABsQAAAAEACwAA
>>>
>>>
>>> AAYAAQAAAC8ACAAMAA0AAQAKAAAAHAABAAAAAAAEsgACrAAAAAEACwAAAAYAAQAAADEAAgAOAAAA
>>>      AgAPAAcAAAAKAAEABQADAAYACA==
>>>
>>>
>>> Of the 2 solutions:
>>>
>>> #1 seems easier to use, but may require more invasive modifications in
>>> the
>>> VM, especially if you want to be able to continue execution after
>>> dumping.
>>>
>> Not sure what #1 really proposes: dumping the complete .jsa archive at
>> runtime or dumping just the loaded classes.
>>
>> If it's just about dumping the loaded class without generating the
>> .jsa archive there's the problem that by default the VM doesn't store
>> the exact bytes of a class after the class was loaded (except when
>> class transformers are registered). So the class files would have to
>> be re-assembled from the internal VM structures (in the same way this
>> is done for class redefinition) and the resulting class-file may be
>> different from the original bytes (i.e. some attributes may be
>> missing).
>
> #1 is for creating the JSA file, not just dumping the class files.
>>
>> If #1 is about creating the whole .jsa archive at runtime (or at VM
>> exit) I think that would be the most attractive solution from a
>> usability point of view although I understand that #2 will be easier
>> to implement in the short term. Regarding the argument that #1 will
>> produce a "binary blob" that's true, but that's already true now when
>> we use "Xshare:dump". I think it should be not to hard to implement a
>> tool based an SA which could introspect a .jsa archive.
>
> The argument about the binary blob is to compare it against the text file
> produced by -XX:DumpLoadedClassList.
>
> One use case to consider is when you have a JAR file that contains several
> apps that each load a unique set of classes. Today, (assuming that custom
> class loaders are not used), you can run each app once with
> -XX:DumpLoadedClassList, and then do an
>
>     cat *.classlist | sort | uniq > combined.classlist
>
> and then create an archive that would work for all these apps.
>

But is this really y relevant use case? Why would I like to create ONE
archive for several apps? This would actually increase the footprint
of a single instance which uses this archive. If I have several apps I
would expect that users create a specific archive for each app to get
the best out of CDS.

> With the binary blob, there's no easy way of doing this. It will be very
> difficult to write a tool to decipher each blob and then somehow combine
> them into a single one.
>

But if users really wants such a "fat" archive, there's a much easier
way: just dump ALL the classes from the .jar file into the archive. A
class list for this could easily be assembled either with an external
tool like cl4cds (or even a simple shell scripts which converts the
output of `unzip -l <jar-file>` into the correct format). Or, even
simpler, by adding a new option to the VM similar to
-XX:DumpLoadedClassList which dumps all the classes it can find on the
class path (and potentially other, configurable locations).

> Thanks
> - Ioi
>
>
>>> #2 would be easier to implement, but the classlist might be huge.
>>>
>>> Also, #2 would allow post-processing tools to remove unneeded classes, or
>>> merge two runs into a single list. The output of #1 is essentially a
>>> binary
>>> blob that's impossible for off-line analysis/optimizations.
>>>
>>>
>>> Any comments, or suggestions for alternatives?
>>>
>>> Thanks
>>> - Ioi
>>>
>>>
>