New proposal for #ReflectiveAccessToNonExportedTypes: Open modules & open packages

[forax at univ-mlv.fr]

Hi Andrew,
welcome to the jungle of the java.lang.invoke API !

----- Mail original -----
> De: "Andrew Dinn" <adinn at redhat.com>
> À: "John Rose" <john.r.rose at oracle.com>
> Cc: "Remi Forax" <forax at univ-mlv.fr>, jigsaw-dev at openjdk.java.net
> Envoyé: Vendredi 11 Novembre 2016 14:43:53
> Objet: Re: New proposal for #ReflectiveAccessToNonExportedTypes: Open modules & open packages

> Hi John,
> 
> I'm reviving an old (by 2-3 weeks) conversation here because it seems
> the right place to ask about how to deal with a disparity I found
> between the use of reflection vs method handles. I'm have retained the
> cc to jigsaw-dev because i) the conversation started there and ii) it
> relates to the question of how well Lookups and MethdoHandles can stand
> in for reflection. Perhaps that's too tenuous to merit a CC but I'm
> adopting the excuse that those who are only interested in more direct
> Jigsaw content can simply ignore the rest of this thread.
> 
> The disparity I am seeing looks like it might just be an argument for
> Maurizio's new reflection API more than anything else but it may just be
> that I am doing something wrong. I'll present the problem in as reduced
> form as I can manage to extract from the Byteman implementation.
> 
> Byteman includes a test to ensure that the type checker allows duck
> typing when a compatible array type is passed as input to a method. So,
> we have a test method
> 
>  public class TestArrayArgTypeCheck extends Test {
>    . . .
>    public void testArrayCall(String[] args)
>      {
>          log("inside testArrayCall");
>      }
>  }
> 
> where log is defined by the parent class Test.
> 
> I use a Byteman rule as follows
> 
>  RULE test array arg type check
>  CLASS TestArrayArgTypeCheck
>  METHOD testArrayCall
>  AT ENTRY
>  BIND test : Test = $this;
>  IF TRUE
>  DO test.log("args : " + java.util.Arrays.asList($args) );
>  ENDRULE
> 
> This rule is 'injected' at the start of the code for method
> testArrayCall -- it's not actually executed as inline bytecode, rather a
> callout to the rule interpreter executes the rule (out of line bytecode
> execution is also an option but I don't [yet?] need to add that
> complexity to the mix). In case you are unfamiliar with how Byteman
> operates I'll summarise its operation.
> 
>  $this and $args name bindings for values passed into the interpreter,
> respectively:
>    the instance of TestArrayArgTypeCheck fielding the call to the
> method (which Byteman knows is of type TestArrayArgTypeCheck)
>    the argument to testArrayCall (which Byteman knows is of type String[])
> 
>  the BIND clause binds a /rule-local/ variable test (of type Test) to
> the instance of TestArrayArgTypeCheck fielding the call to the method
> 
> the DO clause
>   passes $this to Arrays.asList()
>   pastes the result into a String
>   passes the String to Test.log
> 
> So, the test ensures that Byteman's type checker accepts that the
> String[] argument passed to Arrays.asList(Object[]) is a legitimate
> argument without throwing a type exception. It doesn't really matter
> what the log output is but the test does check for an expected output
> and that is where the disparity arises.
> 
> The call to the target method is made by the test code as follows
> 
>  . . .
>  String[] ordinals = { "first", "second", "third" };
>  log("calling testArrayCall");
>  testArrayCall(ordinals);
>  log("called testArrayCall");
>  . . .
> 
> When this is run on JDK8[-] I see this output which matches expectation:
> 
> <log>
> calling testArrayCall
> args : [first, second, third]
> inside testArrayCall
> called testArrayCall
> </log>
> 
> When I run this on JDK9 using a modified version of Byteman that relies
> on MethodHandles I see this output:
> 
> <log>
> calling testArrayCall
> args : [[Ljava.lang.String;@36bc55de]
> inside testArrayCall
> called testArrayCall
> </log>
> 
> So, the String[] array appears to have been wrapped in an Object[]
> before being passed on to Arrays.asList().
> 
> 
> On JDK8[-] I use reflection to execute the method call. So, essentially
> the code looks like this
> 
> class MethodExpression {
>  Method method;
>  Expression recipient;
>  List<Expression> arguments;
>  . . .
>  Object interpret(...)
>    . . .
>    Object recipientValue =
>      (recipient != null
>       ? recipient.interpret(...)
>       : null);
>    int argCount = arguments.size();
>    Object[] argValues = new Object[argCount];
>    for (int i = 0; i < argCount; i++) {
>      argValues[i] = arguments.get(i).interpret(...);
>    }
>    . . .
>    return method.invoke(recipientValue, argValues);
> 
> The essential differences in the code that gets executed on JDK9
> (ignoring that I am inlining code here from different branches) are as
> follows:
> 
> class MethodExpression {
>  Method method;
>  MethodHandle handle = getMethodHandle(method);
>  List<Expression> arguments;
>  . . .
>  Object interpret(...)
>    . . .
>    Object recipientValue =
>      (recipient != null
>       ? recipient.interpret(...)
>       : null);
>    int argCount = arguments.size();
>    Object[] argValues = new Object[argCount];
>    for (int i = 0; i < argCount; i++) {
>      argValues[i] = arguments.get(i).interpret(helper);
>    }
>    . . .
>    if (recipient == null) {
>      handle.invoke(argValues);
>    } else {
>      handle.invokeWithArguments(recipientValue, argValues);
>    }


MH.invokeWithArguments doesn't work like Method.invoke,
a MethodHandle is a function so any invoke* on a method handle is a function call,
there is no method on a method handle that separates the receiver (what you call the recipientValue) from the arguments when performing a call.

MH.invokeWithArguments takes an array of arguments but because it is specified as a varargs you may think that it works like Method.invoke, but it is a trap,
it takes the receiver and the arguments altogether into the same array.


so with a Stream it's something like:
    Object[] argValues = Stream.concat(
        Optional.ofNullable(recipient).map(r -> r.interpret(helper)).stream(),
        Arrays.stream(arguments).map(r -> r.interpret(helper))
      ).toArray(Object[]::new);

    . . .
    handle.invokeWithArguments(argValues);


> 
> regards,
> 
> 
> Andrew Dinn

regards,
Rémi


> -----------
> Senior Principal Software Engineer
> Red Hat UK Ltd
> Registered in England and Wales under Company Registration No. 03798903
> Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander