[Nestmates] 8186763: Nestmate update for lazy validation and virtual invocation

[David Holmes]

Bug: https://bugs.openjdk.java.net/browse/JDK-8186763
webrev: http://cr.openjdk.java.net/~dholmes/8186763/webrev/

These are changes I plan to push as soon as my Committer status is 
confirmed. Details below.

Please note the nestmates prototype is still in a very early form and 
only the actual nestmate tests are known to pass with direct nestmate 
invocations enabled. Many other tests will pass with no problem, but 
there are still issues to resolve in places (CDS) - and we're not quite 
ready to try and build the JDK with direct nestmate invocations enabled.

The nestmate jtreg tests force use of nestmate accesses using 
-XDdisablePrivateAccessors on the @run line. To run other tests with 
direct nestmate invocations add this to the jtreg command-line:

-javacoption:-XDdisablePrivateAccessors

At present the default mode for javac still issues invokespecial. To 
force invokevirtual instead add this to the jtreg command-line:

-javacoption:-XDvirtualizePrivateAccess

Thanks,
David
-----

 From discussions at and after JVMLS, three changes were proposed for 
nestmates:

1. Use invokevirtual for private instance method invocations instead of 
having to make complicated changed to invokespecial.

2. Use invokeinterface for private interface method invocations, instead 
of invokespecial

3. Perform lazy validation of nest membership, including resolution of 
the nest-host, only when a nestmate access check has to be performed.

This update handles items #1 and #3. Those issues are also related in 
that use of invokespecial would trigger nestmate access checks during 
verification, whereas use of invokevirtual does not and so with 
invokevirtual nest membership validation is as late as possible.

--

Details of changes:

src/share/vm/classfile/stackMapFrame.hpp

Remove the previously introduced, but not needed, peek_stack API

src/share/vm/interpreter/linkResolver.cpp

Ensure virtual invocation doesn't find a private superclass method if 
the target method has been removed.
Clean out old experimental code.
Revert the comments re invokespecial usage

src/share/vm/oops/instanceKlass.cpp
src/share/vm/oops/instanceKlass.hpp

Inline the validate_nest_top logic into nest_top so that validation and 
resolution occur when nest_top() is first called - ie when a nestmate 
access check is needed.
Remove the previous call to do validation at link-time.

test/compiler/jvmci/events/JvmciNotifyBootstrapFinishedEventTest.java
test/compiler/jvmci/events/JvmciNotifyInstallEventTest.java
test/compiler/jvmci/events/JvmciShutdownEventTest.java

Undo previous changes that are no longer needed with lazy nestmate 
validation.

test/runtime/Nestmates/membership/ArrayNestTop.jcod
test/runtime/Nestmates/membership/MissingNestTop.jcod
test/runtime/Nestmates/membership/NotAMember.jcod
test/runtime/Nestmates/membership/NotAMember2.jcod
test/runtime/Nestmates/membership/TestNestmateMembership.java
A test/runtime/Nestmates/membership/NotAMember2.java
A test/runtime/Nestmates/membership/PackagedNestTop.java
A test/runtime/Nestmates/membership/PackagedNestTop.jcod
A test/runtime/Nestmates/membership/PackagedNestTop2.java
A test/runtime/Nestmates/membership/PackagedNestTop2Member.jcod
R test/runtime/Nestmates/membership/NestTop.jcod
R test/runtime/Nestmates/membership/NestTop_NestMember.jcod

Rework the nestmate membership tests as they now need to attempt actual 
nestmate invocations to trigger the validation.

test/runtime/Nestmates/privateMethods/ExternalSub.jcod
test/runtime/Nestmates/privateMethods/ExternalSuper.jcod
test/runtime/Nestmates/privateMethods/MissingMethod.jcod
test/runtime/Nestmates/privateMethods/MissingMethodWithSuper.jcod
test/runtime/Nestmates/privateMethods/MissingNestTop.jcod
A test/runtime/Nestmates/privateMethods/TestInvoke.java
A test/runtime/Nestmates/privateMethods/TestInvokeErrors.java
A test/runtime/Nestmates/privateMethods/TestInvokeHierarchy.java
R test/runtime/Nestmates/privateMethods/TestInvokeSpecial.java
R test/runtime/Nestmates/privateMethods/TestInvokeSpecialErrors.java
R test/runtime/Nestmates/privateMethods/TestInvokeSpecialHierarchy.java

Rename all the InvokeSpecial tests/classes/comments to just Invoke - the 
tests are not dependent on, nor necessarily utilise InvokeSpecial. 
(Modulo the one case where verification will fail if using invokespecial.)

A test/runtime/Nestmates/privateConstructors/ExternalSub.jcod
A test/runtime/Nestmates/privateConstructors/ExternalSuper.jcod
A test/runtime/Nestmates/privateConstructors/TestConstructorHierarchy.java

Add some private constructor tests to mimic those for private methods - 
where applicable.