Loading classes with many methods is very expensive

[Peter Levart]

On 10/23/2014 05:06 PM, Joel Borggrén-Franck wrote:
> Hi Martin,
>
> On 23 okt 2014, at 00:53, Martin Buchholz <martinrb at google.com> wrote:
>
>> Here at Google we have both kinds of scalability problems - loading classes
>> from a classpath with 10,000 jars, and loading a single class file packed
>> with the maximal number of methods.  This message is about the latter.
>>
>> If you have a class with ~64k methods with a superclass that also has ~64k
>> methods, class loading that single class will cost you ~30sec and calling
>> Class.getMethods another ~10sec.  Both are unacceptably slow. I think both
>> are due to O(N^2) algorithms, the first in hotspot, and the second in
>> Class.java.
> Throw in lots of interfaces with lots of default methods and i suspect this can get even worse. I spent some time thinking about this when fixing an issue with reflection for default methods, reviewed here [1].

Hi Joel,

I may have found another issue:

interface A4 extends B4, C4 {}
interface B4 extends D4 { void m(); }
interface C4 extends D4 {}
interface D4 { void m(); }

Calling A4.class.getMethods() returns B4.m, D4.m - this is supposed to 
be OK as per JDK 7 spec.

Changing both methods to default methods:

interface A5 extends B5, C5 {}
interface B5 extends D5 { default void m() {} }
interface C5 extends D5 {}
interface D5 { default void m() {}; }

A5.class.getMethods() returns B5.m - this is new in JDK 8 spec for 
default methods.

Now see the following two examples (just one method is default, the 
other is abstract):

interface A6 extends B6, C6 {}
interface B6 extends D6 { void m(); }
interface C6 extends D6 {}
interface D6 { default void m() {}; }

A6.class.getMethods() returns B6.m, D6.m

// B.m, D.m
interface A7 extends B7, C7 {}
interface B7 extends D7 { default void m() {} }
interface C7 extends D7 {}
interface D7 { void m(); }

A7.class.getMethods() returns B7.m


Do last two examples give expected result? Why are A6 and A7 different? 
What is the rule here? I would expect A6.class.getMethods() only return 
the D6.m default method. This is the non-abstract method that gets used 
when implementing an interface.

If A6 example is really showing a bug, then I may have a solution that 
fixes it *and* is faster than current code for getMethods() + it is O(n)...

Is there a test that validates correctness of getMethods() or at least a 
set of interfaces and/or classes to exercise the algorithm and compare 
it to a different implementation?

>
>> I have the start of a fix for Class.java, but it makes the common case
>> slower.  A really good fix is harder to find.  In general, I think
>> Class.java could benefit from some performance-oriented rework.  Is anyone
>> else working on class loading performance, especially in hotspot?
>>
> We have been thinking about replacing the duplication of the method lookup logic in j.l.Class with a call to the VM which should already have the information needed. I’m not sure why the logic was duplicated on the library side way back when this was written. This isn’t something we are actively working on though.

Does VM have enough information to quickly compile the list of public 
methods (getMethods() equivalent) or only to look-up the public method 
(getMethod() equivalent)?


Regards, Peter

>
> As an aside, I often found myself wanting for the actual method descriptor when working with this, but considering there can be *a lot* of instances of Method/Constructor adding a descriptor field to Executable wasn’t an obvious win to me.
>
> cheers
> /Joel
>
> [1] http://mail.openjdk.java.net/pipermail/core-libs-dev/2014-May/026782.html