[Nestmates] RFR: 8189163: [Nestmates] Updated invokeInterface selection/resolution rules

[David Holmes]

Bug: https://bugs.openjdk.java.net/browse/JDK-8189163
webrev: http://cr.openjdk.java.net/~dholmes/8189163/webrev/
VM Spec update: 
http://cr.openjdk.java.net/~dholmes/8010319/specs/JVMS/nestmates.html

The "resolution" part of this is actually already addressed by virtue of 
being able to use invokeinterface for private interface methods.

This part addresses the key spec change in relation to existing method 
selection rules for invokeinterface - which is intended to bring it into 
line with how invokevirtual already behaves. The key points to note in 
method selection are:

1. If the resolved method is private then the selected method is always 
the resolved method. (And that is always the implementation in the 
interface when dealing with invokeinterface.)

2. A private method never overrides any method by the same 
name/signature in the inheritance hierarchy.

The key consequence of (2) is that private methods are skipped when 
selecting the method to call when the resolved method is not private. So 
in the past, for example, if you were looking for an implementation of a 
public interface method m() in a class C, and you found a private 
definition of m(), you would "select" it and then throw 
IllegalAccessError. Under the new rules you skip-over/ignore it and so 
continue looking for a suitable m() implementation.

In terms of implementing this change there were three very similar and 
very small changes to cause private methods to be skipped:

1. src/hotspot/share/interpreter/linkResolver.cpp: 
runtime_resolve_interface_method

More appropriately named "select_interface_method", in this method we 
now call LinkResolver::lookup_instance_method_klasses with the 
"skip_private" flag.

The bulk of the changes in LinkResolver and src/hotspot/share/oops/* are 
simply to expose the PrivateLookupMode parameter so that it can be 
passed through explicitly to uncached_lookup_method and 
find_instance_method. By default we assume "find_private", so we did not 
have to modify all the existing callsites.

2.  src/hotspot/share/oops/klassVtable.cpp: 
klassItable::initialize_itable_for_interface

When initializing the itable for a class, skip private class methods as 
they can never be the implementation of an interface method.

3.  src/hotspot/share/code/dependencies.cpp: bool is_witness(Klass* k)

When looking for an implementation for "m" in k to see if the uniqueness 
of "m" has changed, skip private methods as they cannot override the 
known "m" and so do not affect its uniqueness.

The change in src/hotspot/share/prims/methodHandles.cpp corrects a 
terminology issue whereby a private interface method is referred to as a 
"default method", which it is not.

Tests:

There are two new tests:

- runtime/Nestmates/methodSelection/TestInterfaceMethodSelection.java
- runtime/Nestmates/methodSelection/TestMethodSelection.java

Although only invokeinterface selection rules changed, I wrote the 
equivalent invokevirtual version as a sanity check. The tests start from 
a known resolved method and then tests the selection process for 
private/public variants of a method in a hierarchy of 2 or 3. There is 
very elaborate documentation in the tests explaining exactly what they 
do (and don't do). The obvious variants of these tests with different 
resolution and accessibility are already tested elsewhere, so I did not 
duplicate that effort.

The following existing tests had to be modified:

- compiler/jsr292/methodHandleExceptions/TestAMEnotNPE.java

The cases which previously selected a private class method and then 
threw IllegalAccessError, no longer do that. Whether a test case results 
in a successful call or an exception depends on the overall structure of 
the test case.

- runtime/SelectionResolution/classes/selectionresolution/Template.java

As discussed in the open issue:

https://bugs.openjdk.java.net/browse/JDK-8194857

the selectionResolution tests need to be updated for the changes 
delivered in Nestmates. Unfortunately they are very difficult tests to 
refactor and so at this stage the failing tests have been excluded. In 
this particular case I was able to make an adjustment at the second 
level in the Template IfaceMethodrefSelectionOverrideNonPublic by 
commenting out all the cases involving private interface methods. That 
meant that, compared to commenting out the top-level tests, instead of 
reducing the test count by 7800, it only reduced by 2600.

Testing:
  - jtreg hotspot/runtime
          hotspot/compiler
  - hs-tier1-2
  - jdk tier1-3
  - internal resolution/selection tests

Thanks,
David