Bug in HS interpreter: invokeinterface calls non-public method

[Vladimir Parfinenko]

Hi all,

I think I have found a bug in HotSpot interpreter.

The problems happens while invokeinterface of public method from
java.lang.Object (e.g. hashCode()) in case when the actual method
implementation is non-public (e.g. protected).

JVMS tells the following about invokeinterface instruction:
Otherwise, if step 1 or step 2 of the lookup procedure selects a method
that is not public, invokeinterface throws an IllegalAccessError.

However in some cases HS interpreter ignores this access check and
invokes non-public method.

Minimal example using jasm from asmtools is attached below. Compiling
and running it gives the following:

  $ jasm BadImpl.jasm && javac Caller.java
  
  $ java -Xint Caller
  Should pass:
  Should throw IAE:
  Exception in thread "main" java.lang.RuntimeException: protected
hashCode was called
          at BadImpl.hashCode(BadImpl.jasm)
          at Caller.main(Caller.java:11)
  
  $ java -Xcomp Caller
  Should pass:
  Should throw IAE:
  Exception in thread "main" java.lang.IllegalAccessError:
BadImpl.hashCode()I
          at Caller.main(Caller.java:11)


Note that first invocation ("Should pass") is necessary to reproduce the
problem. If you remove it everything works as expected.

Regards,
Vladimir Parfinenko


----------------------- Caller.java -----------------------
  public class Caller {
      public static void main(String[] args) {
          Interf x;
  
          System.out.println("Should pass:");
          x = new GoodImpl();
          x.hashCode();
  
          System.out.println("Should throw IAE:");
          x = new BadImpl();
          x.hashCode();
      }
  }
  
  interface Interf {
      @Override
      int hashCode();
  }
  
  class GoodImpl implements Interf {
  }
----------------------- Caller.java -----------------------

----------------------- BadImpl.jasm -----------------------
  super class BadImpl
  	implements Interf
  {
  
  
  Method "<init>":"()V"
  	stack 1 locals 1
  {
  		aload_0;
  		invokespecial	Method java/lang/Object."<init>":"()V";
  		return;
  }
  
  // override of Object method with protected one, javac doesn't allow
this
  protected Method hashCode:"()I"
  	stack 3 locals 1
  {
  		new	class java/lang/RuntimeException;
  		dup;
  		ldc	String "protected hashCode was called";
  		invokespecial	Method
java/lang/RuntimeException."<init>":"(Ljava/lang/String;)V";
  		athrow;
  }
  
  }
----------------------- BadImpl.jasm -----------------------