[Nestmates] RFR: 8187536: [Nestmates] Assertion failure when the compiler thread appears to trigger classloading

[David Holmes]

bug: https://bugs.openjdk.java.net/browse/JDK-8187536
webrev: http://cr.openjdk.java.net/~dholmes/8187536/webrev/

The compiler thread can hit a path that requires method resolution, 
which leads to an access check, which leads to a nestmate access check, 
which triggers the resolving (and possibly loading) of the nest-top 
class. That hit an assert at the top of the class-loading code in 
SystemDictionary::resolve_or_null. In most cases the nest-top class is 
already loaded, it just hasn't been resolved into the nest-top field of 
the nestmate being checked, so there is no requirement to abort when a 
compiler thread takes this initial path.

This partial solution pushes the assertion deeper into the classloading 
mechanics to point where it is more likely that classloading actually 
may have to occur.

This general issue needs closer examination as it may be that there can 
still be cases in which we actually do have to load a nest-top class, 
and the compiler thread must not cause an abort in that case, but 
potentially bail-out of compilation temporarily so the loading can be 
done in the interpreter.

The current fix removes a roadblock to being able to build the JDK with 
the new nestmate logic enabled permanently.

The run of a single test using -Xcomp is enough to validate the fix for now.

Thanks,
David
-----