Improving AppCDS for Custom Loaders

[Ioi Lam]

PROBLEM:

As discussed with Volker and Yumin in previous e-mails, AppCDS has some 
experimental support for custom class loaders. However, it's not very 
easy to use.

For example, you can write a classlist like this:

     java/lang/Object id: 1
     CustomLoadee id: 2 super: 1 source: /tmp/foo.jar

The CustomLoadee class will be stored in the shared archive with a CRC 
code. During runtime, if a customed loader wants to load a class of the 
same name, and its classfile has the same size and CRC as the archived 
class, the archived version will be loaded. This speeds up class loading 
by avoiding parsing the class file, and saves space by sharing the 
mmap'ed class metadata across processes.

You can see an example test at:

http://hg.openjdk.java.net/jdk/hs/file/46dc568d6804/test/hotspot/jtreg/runtime/appcds/customLoader/HelloCustom.java

However, the current scheme requires you to specify all the super 
classes and interfaces. There's no support provided by the 
-XX:DumpLoadedClassList option.It can be helped somewhat with Volker's 
tool: https://github.com/simonis/cl4cds


POSSIBLE SOLUTIONS:

1. "Dump-as-you-go". As suggested by Yumin, we can provide a jcmd to ask 
a running JVM process to dump all of its loaded classes, including those 
loaded by custom loaders, into an archive. An alternative is to dump the 
archive at JVM exit time (or when you press Ctrl-C, etc.

2. Add information about the custom classes for -XX:DumpLoadedClassList. 
The trouble is some class loaders don't specify a code source that can 
be understood by the built-in class loaders. For example, the "Fat Jars" 
would have a code source like

jar:file:/jdk/tmp/test-1.0-SNAPSHOT.jar!/BOOT-INF/lib/validation-api-2.0.1.Final.jar!/

also, many custom loaders would pre-process the classfile data before 
defining the class, so we can't simply archive the version of the class 
on disk.

One possible solution for #2 is to include the class file data in the 
-XX:DumpLoadedClassList output:


java/lang/Object id: 1
     CustomLoadee id: 2 super: 1 source: base64
yv66vgAAADQAFwoABAAQCQAFABEHABIHABMHABQBAAJIaQEADElubmVyQ2xhc3NlcwEABjxpbml0
PgEAAygpVgEABENvZGUBAA9MaW5lTnVtYmVyVGFibGUBAAFmAQADKClJAQAKU291cmNlRmlsZQEA
CkhlbGxvLmphdmEMAAgACQwAFQAWAQAFSGVsbG8BABBqYXZhL2xhbmcvT2JqZWN0AQAISGVsbG8k
SGkBAAFYAQABSQAhAAMABAAAAAAAAgABAAgACQABAAoAAAAdAAEAAQAAAAUqtwABsQAAAAEACwAA
AAYAAQAAAC8ACAAMAA0AAQAKAAAAHAABAAAAAAAEsgACrAAAAAEACwAAAAYAAQAAADEAAgAOAAAA
     AgAPAAcAAAAKAAEABQADAAYACA==


Of the 2 solutions:

#1 seems easier to use, but may require more invasive modifications in 
the VM, especially if you want to be able to continue execution after 
dumping.

#2 would be easier to implement, but the classlist might be huge.

Also, #2 would allow post-processing tools to remove unneeded classes, 
or merge two runs into a single list. The output of #1 is essentially a 
binary blob that's impossible for off-line analysis/optimizations.


Any comments, or suggestions for alternatives?

Thanks
- Ioi