Suggestion: allow accessible reflection on protected methods of exported types.

[Jochen Theodorou]

On 09.01.2017 12:47, Andrew Dinn wrote:
[...]
> If you carefully review discussion on the subject of this (slightly
> false) dichotomy between static and dynamic capabilities I think you
> will find that there has been a lot of explicit discussion. In my view,
> your summary of the status quo is at best out of date and at worst
> misrepresents the goals of the Jigsaw project and the wider goals of the
> OpenJDK project as a whole.
>
> There is indeed a desire to limit opportunities for programs to rely on
> reflection.

The problem is simply with old reflective code. You tend to have - and I 
have seen that in many applications - a kind of central part doing 
invocation for you. You could call it a reflective runtime. For this 
kind of code doing rights via @CallerSensitive tends to be unusable, 
because you tend to have code that wraps the call itself. And once you 
have that, it is the rights of your runtime, not that of the calling 
code. It was simply not "normal" to do, what you can now do with 
MethodHandles, which is returning something, which will then be invoked 
instead of doing the invocation right away. And considering cases where 
you for example need the post processing of exceptions and such I don´t 
really see how you could have done that with reflection anyway.

Now having these "central invokers" means you have to work completely 
out of context as far @CallerSensitive is considered. And that means you 
are forced to use setAccessible, if you need to handle more than just 
public class members (or public class containers for that matter). And 
this kind of code is now practically crippled.

And as you wrote, that is exactly what was intended to be done.

But that means, that language runtimes tend to have a problem now if 
they depend on such central invoker patterns.

I won´t deny that maybe in some structures things got a bit out of hand. 
But fixing them after years of working without breaking changes is near 
impossible.

That is because I cannot simply change exported API to reach through 
Lookup objects, since they would be a breaking API change. Plus the 
Lookup objects can only do what their context allows them to do, they do 
not automatically give me the right to invoke methods on other objects I 
normally have no access rights for in that context. I would need the 
Lookup object for the goal, which might not be obtainable. And again, 
that is exactly what was intended to be broken.

example... Let us say you work with a GroovyIntereptable:

class A implements GroovyIntercepable {
   def invokeMethod(String name, args) {
    ...
   }
}

class B extends A {
   private someMethod() {}
}

for every method call on an instance of A or B (for example 
b.callSomemethod()) the invokeMethod method is called. I was now 
perfectly able to invoke someMethod on instances of B in the 
invokeMethod method of A by simply doing this.someMethod(). I assume the 
code will still work if B is in the same module as A, but if not, then 
this code will no longer work using central invoker patterns. Of course 
I could now claim that B may expose a helper method of some kind, which 
is automagically called and delegates to someMethod, or that I can 
replace the usage by MethodHandles to some extend. But if class B is 
written in Java, then I fail with the first approach and if the actual 
handling is not done in A, but in a delegate of A, I fail with the 
MethodHandles approach behind the scenes as well. Of course I could have 
the Lookup object in this case. But point being, the API does not allow 
for it without breaking changes.

Then how else can I add such a helper method for my runtime? Agents, as 
Remi said? Can agents add methods to an already loaded class? I am 
unaware of this being possible now. Which means I have to start an agent 
on VM startup, which means a more complicated command line for the user, 
but it means first and for all it counters the usage of Groovy as 
non-intrusive library. Right now, if I do not use anything from Groovy, 
the Groovy runtime won´t be involved. With having to have an agent to 
transform classes on load, this is changed.

In other words, we are destroying one feature to rescue another. And I 
do not see a good solution here, I doubt there can really exist one, 
because it is the very goal of jigsaw to "limit opportunities for 
programs to rely on reflection". And just to make this 100% clear... 
even the invokedynamic based version of Groovy has to use reflection to 
some degree to get everything running. Basically all those things that 
jigsaw forbids now.

> That desire is linked to the motivation for implementing
> Jigsaw. Reflection as it was provided in JDK8- does not really fit well
> with /the/ main goal of modularization i.e. ensuring that module
> internals -- in particular, JDK runtime internals -- can be guaranteed
> to remain private to client code. The concern is wider than simply
> meeting this Jigsaw requirement. That guarantee is critical to ensuring
> that module implementations, most especially JDK runtime modules
> implementations, can be maintained, extended and optimized without
> breaking backward compatibility. Some breakage now is being traded off
> against breakage later.

"some" is relative. For me it means to rewrite the extensible object 
protocol for Groovy (MOP). All signatures of all extension points will 
change. And even then I cannot do all I did before, because before I was 
able to call methods you ordinarily have no access to in a Java world 
without reflection.

> This change to reflective accessibility is not simply a wholesale
> removal of capability.It is being balanced by a mix of completed and
> in-progress changes which i) allow suitably privileged code (such as
> agents or containers) to enable reflective access where they see fit and
> ii) provide an alternative mechanism to achieve the same end result as
> reflection but instead based on method handles. The latter alternative
> is especially important as it will ensure more precisely controlled and
> recorded access and, as one very important consequence, enable better
> performance from the compiler(s).

that kind of balancing is a POV depending action you know. For Java this 
is simply a completely different story than it is for Groovy. 
Accordingly it must look unbalanced for Groovy. Additionally I do not so 
(i). How can a library add reflective access where they see fit in a 
non-intrusive way? And (ii) is simply not there. For example... can you 
call Object#clone on Object using MethodHandles? I actually did not try 
this, but since the method is protected and in a module I would assume 
no. You had been able to do so by reflection. Let us not discuss about 
if that makes sense or not. It just shows for me that (ii) is simply not 
there.

They cannot be there, because if you could do everything you could have 
done with reflection in a jigsaw world, it means you can bypass the 
module system. As such it would counter the design goal to have stronger 
encapsulation.

And on the performance of compilers... that part you have to explain to me.

> There have also been many other changes made to ensure that Jigsaw
> supports other important aspects of dynamic programs. the Layer API in
> particular has been extended significantly to allow greater control over
> loading and linking of dynamically deployed modules, including support
> for undeploying and redeploying modular code (as required by e.g. EE
> containers). You might want to look over the archives of discussions on
> the JSR expert group, observers and comments mail lists to review all
> these proposed changes in full or maybe even sign up to the last of
> these lists in order to monitor progress on work that still remains to
> be completed.
>
>    http://mail.openjdk.java.net/pipermail/jpms-spec-experts/
>    http://mail.openjdk.java.net/pipermail/jpms-spec-observers/
>    http://mail.openjdk.java.net/pipermail/jpms-spec-comments/

How do layers help me in calling a method like Object#clone? I can see 
that if you are doing a container of sorts or module system like osgi or 
maybe the jboss modules, that there might be some benefit by jigsaw and 
that it may indeed give you more control in certain places.

But for a simple command line script this simply does not matter.

bye Jochen