Proposal: Fully Concurrent ClassLoading

[David M. Lloyd]

Ah, well for our concurrent class loader we just keep simple counters 
for all that stuff.  If we can boost that up to the ClassLoader level 
that'd be nifty.  The stats we track that have been useful are:

- Number of defined classes
- Number of 'races' i.e. define when already existent
- Number of modules (== class loaders) created in the module loader
- CPU time spent defining classes (though this is currently hard to 
capture accurately)

The platform ClassLoadingMXBean has a couple useful stats already; I 
could see adding a counter for the number of duplicate defines, and 
maybe adding some basic timing information in there too.  One other 
metrics that could be useful: the number of failed class load attempts, 
number of failed due to being missing, and the number failed due to 
linkage problems.

On 12/11/2012 07:32 PM, David Holmes wrote:
> On 12/12/2012 1:10 AM, David M. Lloyd wrote:
>> No problem. What do you mean by "instrumentation"?
>
> Any means of "seeing" when parallel define class actually occurred:
>
> -  -verbose:class logging ?
> -  jvmstat counter ?
> - -XX:+TraceParallelDefine ?
> - ???
>
> Thanks,
> David
>
>> On 12/10/2012 11:18 PM, David Holmes wrote:
>>> David,
>>>
>>> Many thanks for trialling this so promptly!
>>>
>>> Do you have any suggestions for what instrumentation you would like to
>>> see accompany this?
>>>
>>> David
>>>
>>> On 11/12/2012 1:41 PM, David M. Lloyd wrote:
>>>> On 12/10/2012 06:36 PM, David Holmes wrote:
>>>>> On 10/12/2012 11:53 PM, David M. Lloyd wrote:
>>>>>> On 12/09/2012 10:03 PM, David Holmes wrote:
>>>>>>> That sounds promising. Are you in a position to try out this
>>>>>>> proposal on
>>>>>>> a testbed with your app server?
>>>>>>
>>>>>> Sure, I'd love to. What tree should I build?
>>>>>
>>>>> Please apply the patches from the webrevs here:
>>>>>
>>>>> http://cr.openjdk.java.net/~dholmes/concurrent-loaders/webrev.hotspot/
>>>>> http://cr.openjdk.java.net/~dholmes/concurrent-loaders/webrev.jdk/
>>>>>
>>>>> They should apply to a jdk8 or tl forest. (And I hope I didn't mess
>>>>> something up generating the webrev ;-) )
>>>>
>>>> Well, I've just gotten everything working and done some cursory testing
>>>> on a startup of an "empty" JBoss AS 7 instance, and then reviewing the
>>>> metrics reported by our class loader. Our timing metrics are not really
>>>> great for general-purpose benchmarking (for various uninteresting
>>>> reasons), but I can at least get enough information to know everything
>>>> is working more or less as expected:
>>>>
>>>> On JDK 6 with our "unsafe" lockless modification, we're typically
>>>> loading 4707 classes, and I'm seeing anywhere from 0 to 5 define races
>>>> that were automatically resolved.
>>>>
>>>> On JDK 7 using the standard registerAsParallelCapable() mechanism, we
>>>> are loading 4711 classes and I'm seeing 35-50 define races that were
>>>> automatically resolved - the overhead of locking starts to come to the
>>>> fore I think.
>>>>
>>>> On JDK 8 with your patches, we are loading around 4750 classes and
>>>> there
>>>> are, as expected, 0 define races (I believe, however, that we're
>>>> getting
>>>> a false count though whenever defineClass() returns an already-defined
>>>> class - it would be nice if there were *some* way to detect that this
>>>> happened).
>>>>
>>>> On our class loader implementations, I'm initializing this way:
>>>>
>>>> static {
>>>> try {
>>>> ClassLoader.registerAsFullyConcurrent();
>>>> } catch (Throwable ignored) {
>>>> try {
>>>> ClassLoader.registerAsParallelCapable();
>>>> } catch (Throwable ignored2) {
>>>> }
>>>> }
>>>> }
>>>>
>>>> The debugging message confirms that our class loaders are successfully
>>>> registered as fully concurrent, and the fact that it doesn't hang or
>>>> crash on JDK 7 indicates that they are still properly being registered
>>>> as parallel-capable on that platform.
>>>>
>>
>>


-- 
- DML