Should setAccessible be part of Java or not? (was Re: It's not too late for access control)

[Alex Buckley]

On 7/15/2016 6:21 AM, Jochen Theodorou wrote:
> I will give you an analysis of my situation so far:
>
> assuming the whole Groovy runtime is a module, and assuming I have other
> modules as well, and then I want to use a Groovy script at runtime, to
> access those modules. And let us assume those modules do not know of
> Groovy. Let us further assume the Groovy runtime will be in a loader
> that delegates to a loader knowing the classes of those modules, or is
> in the same loader.
>
> So first problem is ... can I use the unnamed module for the scripts?
> They read every (named) module, thus they can access the types.
> Next are 4 types of method invocations to consider for the case of  an
> invocation from script in the unnamed module to named module:
> reflective, indy, direct, generated bytecode for callsites (I name this
> one csgen in short). The method itself is looked up by reflection
>
> * for indy it is only a question of the lookup object, which will be
> provided by the script and can do the call
> * direct calls will work
> * reflection would be done from the groovy runtime, thus the runtime
> need to have a read on the named module. That´s the first time I will
> need Java9 special code in this scenario
> * csgen is more unclear. Let us assume it is in the unnamed package as
> well, in a loader that delegates to the script loader. Basically it
> becomes a direct call and works.
>
> Next step is for my Groovy runtime having to call a method from the
> script. Since everything is exported there is no problem on this side.
> * indy will work after adding the read
> * reflection - same
> * direct call - same
> * cs gen... the call to the csgen method will be done by an
> interface/class of the Groovy runtime, and since the csgen class is in
> the unnamed package, it has no problem calling the script methods.

With one exception, everything above is correct. (Exception: reflection 
from the Groovy module to a named module, performed on behalf of the 
script, does NOT need Java 9 code, because reflection gets readability 
for free. Also, you mean that csgen code is in the unnamed MODULE; its 
package is not so important.)

Key point: Just because a framework is delivered as a module, does not 
mean the framework has to change its class loading behavior or require 
that its plugins/scripts/introspectees be delivered as modules. Groovy 
can continue to load scripts (or classes derived therefrom) into 
user-defined loaders. The scripts can continue to access each other 
(assuming suitable loader delegation) and Java platform APIs and Groovy 
runtime APIs (assuming the Groovy module exports some packages).

> Let us look at the same scenario with a named module for scripts. I will
> use that as a step in between to think about actual compile time modules
> written in Groovy. So for simplicity I assume I can create such a module
> at runtime and it will export everything. How to do that? I have not the
> slightest idea. Anyway, that means now I have calls from named script
> module to named module:
>
> * indy will work after the script module added a read to the target
> module. This will require in the script, with a direct call to addReads,
> because of caller sensitivity. Which means instead of just having JDK9
> specific code in my runtime, I will now need JDK9 specific code to my
> bytecode as well. And this needs to happen before any normal method
> invocation can take place, thus static initializer, first thing... which
> means completely new code for the compiler, that will work only on JDK9.
> This means the compiler will then have to at least know if he compiles
> for JDK9 or not...
> * direct call - same
> * reflection - the call is effectively done from the Groovy runtime, so
> the Groovy runtime will have to add the read.
> * csgen in the unnamed module means.. since the call is gated by an
> interface/class from the runtime I assume I do not need a read edge from
> the script module to the unnamed module csgen is in. The csgen class
> then will be able to read the exported class of the target module, thus
> has no problem.
>
> Calls from the groovy runtime to the script are not different to before.

For creating named modules at run time, see the javadoc for 
java.lang.reflect.Layer and all the linked javadoc down to 
java.lang.module.ModuleDescriptor.Builder.

You're right about indy calls and direct calls following readability, so 
you can either have the script's module require another named module in 
the Configuration that you pass to the Layer, or you can have the 
script's own code add readability as you describe.

(For reflection, note again that the Groovy runtime does NOT have to add 
a read from its module to another named module.)

Now for a pivot. All the issues you raise in the rest of this email, 
about how named modules containing bytecode-derived-from-Groovy-source 
interact with named modules of the Groovy runtime or otherwise, are 
familiar. Why? Because Nashorn faced exactly the same issues. There will 
be a presentation at the JVM Language Summit (jvmlangsummit.com) on how 
Nashorn uses modules to encapsulate its generated bytecode. The video 
will be online at YouTube the next day. It'll make discussing the issues 
below much easier.

Alex

> Next step is then precompiled Groovy code in a named module. The
> difference to before is that we now have to look at calls from hidden
> API to hidden API of the same module and we have compile time defined
> requires and exports.
>
> Since the Groovy runtime needs to be able to call some methods by
> reflection all packages, including the hidden API, will have to be
> exported to the groovy runtime and of course there will be a require for
> the Groovy runtime.
>
> Let us briefly look at a call from the script module to an exported
> class in another named module again... the script module will now
> require the other named module, which means we have a reads here already.
> * direct call works
> * reflection - runtime has to add read
> * indy works
> * csgen in the unnamed module... the call is again done using
> interfaces/classes from the runtime, meaning it should work.
>
> Then let us look at a call from hidden/public API to hidden API of the
> same module of our precompiled script.
> * direct call works
> * indy works
> * reflection works, since everything is exported to the groovy runtime..
> * csgen in the unnamed module... while the call to csgen will work
> (gated by classes/interfaces of the runtime), the call from csgen to
> hidden API will fail. Adding a read won´t help, end of line. A named
> module would have the same problem. Solution unclear.
>
> As for calls from the groovy runtime to the hidden API of the script
> module... Some csgen problem, everything else is fine.
>
> Finally calls from the script module to the hidden API of another
> module... I can make this sometimes work with indy and with reflection.
> csgen will have the known problem, direct calls will not work. Of course
> there is now the question of if I need such a call... assume you did
> write a program like this:
>
> Module A, export exported
> package exported
> class MyExportedList extends ArrayList {
>    def sum(init){
>      def ret = init
>      for (it in this) ret +=it
>      return ret
>    }
> }
>
> Module B, hidden API, require A
> class HiddenX {
>    int value
>    def plus(HiddenX other) { return value+other.value }
> }
>
> def list = [new HiddenX(value:1), new HiddenX(value:2), new
> HiddenX(value:3)] as MyExportedList
> assert list.sum(new HiddenX(value:0)) == 6
>
> This will require the sum method in Module A to call plus on HiddenX for
> the "+=". There is no interface or class for this, just the public
> method. Since the class is in the hidden API of B, A will not be able to
> access it.
> * A direct method call is out of question for this of course, so no
> static case at all.
> * reflection through the groovy runtime
> * indy... before I would have used the lookup object of the caller to
> realize the call. Since A cannot read HiddenX, this won´t work anymore.
> I would have to resort to a private "everything is allowed" constructor
> in Lookup. Basically a hack, of which I won´t know if it will work in
> the next JDK version. Maybe I could "steal" a lookup object from HiddenX
> to avoid this, but it is very allowed to write HiddenX in another
> language like Java, and then I will not be able to produce a helper
> method for this. Meaning I will have to stay with the hackish
> maintenance nightmare of using a private constructor.
> * csgen... the sad story continues.
>
> And btw, how am I supposed to decide if this call is allowed or not?
>
> To sum things up:
> * module runtime creation and handling beyond adding reads is unclear
> * setAccessible failing on public classes from hidden APIs is
> inconvenient, but at least we can find code for this, which is
> compatible for multiple JDK versions
> * csgen will probably have to fall back to reflection for most cases,
> which means it will be a lot slower. A bigger change would be to give it
> an indy backend. We would probably still suffer a performance penalty,
> since we then have an additional layer with wrapping in between, that
> cannot be removed by the JIT, as long as the JIT is unable to look
> beyond the indy callsite itself.
> * the indy part will have to use "forbidden" API now.
> * runtime scripts will go into the unnamed module
> * hidden APIs and non-public modifiers will be ignored by Groovy for as
> long as the "forbidden" API works, which keeps current semantics
> * most of the groovy runtime makes still heavy use of reflection and
> will probably have to be reworked to use indy everywhere, if on JDK9
> (probably JDK8+).
>
> And I am not happy with this outcome so far.
>
> bye Jochen