Should setAccessible be part of Java or not? (was Re: It's not too late for access control)

[Jochen Theodorou]

On 14.07.2016 23:24, Alan Bateman wrote:
>
>
> On 14/07/2016 21:20, Jochen Theodorou wrote:
>>
>> What I would wish for at this moment is a document explaining the
>> runtime aspects of the module system, including limitations reflection
>> and normal method calls, as well as things like layers / runtime
>> generated modules - as well as the methods to be used to create and
>> change these. I don´t anything JLS ready here and just the behaviour
>> of the one of the latest releases is also good. I cannot give feedback
>> on a design I do not know. Because maybe some frameworks are content
>> with the compile time side of this, I am not.
>>
> Jochen - have you worked through the documents, slides and recordings
> that are linked from the main Project Jigsaw page [1]?
>
> I know in other threads we we alluded to slides on how
> Nashorn/Javascript has been updated to work with modules (and spin
> modules at runtime). I realize Groovy is different but I think we should
> do this as it may be useful to other language implementers too.
>
> [1] http://openjdk.java.net/projects/jigsaw/

Well, I was hoping for a document, videos do not work well for me and 
slides are better, but often useless without the actual talk... depends 
of course

The problem is that this is a lot of material that barely scratches the 
points I am wondering about.

Example http://openjdk.java.net/projects/jigsaw/talks/

Javaone 2015
Prepare for JDK 9 -> for me useless
Introduction to Modular Development -> for runtime aspects useless
Advanced Modular Development -> mentions addReads on slide 36 and is 
done with the runtime aspect with that basically.
Project Jigsaw: Under the Hood -> is more interesting from slide 34 on, 
but does not answer my questions enough.
Project Jigsaw Hack Session -> the whole thing as video

Devoxx BE 2015 is only video

I will give you an analysis of my situation so far:

assuming the whole Groovy runtime is a module, and assuming I have other 
modules as well, and then I want to use a Groovy script at runtime, to 
access those modules. And let us assume those modules do not know of 
Groovy. Let us further assume the Groovy runtime will be in a loader 
that delegates to a loader knowing the classes of those modules, or is 
in the same loader.

So first problem is ... can I use the unnamed module for the scripts? 
They read every (named) module, thus they can access the types.
Next are 4 types of method invocations to consider for the case of  an 
invocation from script in the unnamed module to named module: 
reflective, indy, direct, generated bytecode for callsites (I name this 
one csgen in short). The method itself is looked up by reflection

* for indy it is only a question of the lookup object, which will be 
provided by the script and can do the call
* direct calls will work
* reflection would be done from the groovy runtime, thus the runtime 
need to have a read on the named module. That´s the first time I will 
need Java9 special code in this scenario
* csgen is more unclear. Let us assume it is in the unnamed package as 
well, in a loader that delegates to the script loader. Basically it 
becomes a direct call and works.

Next step is for my Groovy runtime having to call a method from the 
script. Since everything is exported there is no problem on this side.
* indy will work after adding the read
* reflection - same
* direct call - same
* cs gen... the call to the csgen method will be done by an 
interface/class of the Groovy runtime, and since the csgen class is in 
the unnamed package, it has no problem calling the script methods.

Let us look at the same scenario with a named module for scripts. I will 
use that as a step in between to think about actual compile time modules 
written in Groovy. So for simplicity I assume I can create such a module 
at runtime and it will export everything. How to do that? I have not the 
slightest idea. Anyway, that means now I have calls from named script 
module to named module:

* indy will work after the script module added a read to the target 
module. This will require in the script, with a direct call to addReads, 
because of caller sensitivity. Which means instead of just having JDK9 
specific code in my runtime, I will now need JDK9 specific code to my 
bytecode as well. And this needs to happen before any normal method 
invocation can take place, thus static initializer, first thing... which 
means completely new code for the compiler, that will work only on JDK9. 
This means the compiler will then have to at least know if he compiles 
for JDK9 or not...
* direct call - same
* reflection - the call is effectively done from the Groovy runtime, so 
the Groovy runtime will have to add the read.
* csgen in the unnamed module means.. since the call is gated by an 
interface/class from the runtime I assume I do not need a read edge from 
the script module to the unnamed module csgen is in. The csgen class 
then will be able to read the exported class of the target module, thus 
has no problem.

Calls from the groovy runtime to the script are not different to before.

Next step is then precompiled Groovy code in a named module. The 
difference to before is that we now have to look at calls from hidden 
API to hidden API of the same module and we have compile time defined 
requires and exports.

Since the Groovy runtime needs to be able to call some methods by 
reflection all packages, including the hidden API, will have to be 
exported to the groovy runtime and of course there will be a require for 
the Groovy runtime.

Let us briefly look at a call from the script module to an exported 
class in another named module again... the script module will now 
require the other named module, which means we have a reads here already.
* direct call works
* reflection - runtime has to add read
* indy works
* csgen in the unnamed module... the call is again done using 
interfaces/classes from the runtime, meaning it should work.

Then let us look at a call from hidden/public API to hidden API of the 
same module of our precompiled script.
* direct call works
* indy works
* reflection works, since everything is exported to the groovy runtime..
* csgen in the unnamed module... while the call to csgen will work 
(gated by classes/interfaces of the runtime), the call from csgen to 
hidden API will fail. Adding a read won´t help, end of line. A named 
module would have the same problem. Solution unclear.

As for calls from the groovy runtime to the hidden API of the script 
module... Some csgen problem, everything else is fine.

Finally calls from the script module to the hidden API of another 
module... I can make this sometimes work with indy and with reflection. 
csgen will have the known problem, direct calls will not work. Of course 
there is now the question of if I need such a call... assume you did 
write a program like this:

Module A, export exported
package exported
class MyExportedList extends ArrayList {
   def sum(init){
     def ret = init
     for (it in this) ret +=it
     return ret
   }
}

Module B, hidden API, require A
class HiddenX {
   int value
   def plus(HiddenX other) { return value+other.value }
}

def list = [new HiddenX(value:1), new HiddenX(value:2), new 
HiddenX(value:3)] as MyExportedList
assert list.sum(new HiddenX(value:0)) == 6

This will require the sum method in Module A to call plus on HiddenX for 
the "+=". There is no interface or class for this, just the public 
method. Since the class is in the hidden API of B, A will not be able to 
access it.
* A direct method call is out of question for this of course, so no 
static case at all.
* reflection through the groovy runtime
* indy... before I would have used the lookup object of the caller to 
realize the call. Since A cannot read HiddenX, this won´t work anymore. 
I would have to resort to a private "everything is allowed" constructor 
in Lookup. Basically a hack, of which I won´t know if it will work in 
the next JDK version. Maybe I could "steal" a lookup object from HiddenX 
to avoid this, but it is very allowed to write HiddenX in another 
language like Java, and then I will not be able to produce a helper 
method for this. Meaning I will have to stay with the hackish 
maintenance nightmare of using a private constructor.
* csgen... the sad story continues.

And btw, how am I supposed to decide if this call is allowed or not?

To sum things up:
* module runtime creation and handling beyond adding reads is unclear
* setAccessible failing on public classes from hidden APIs is 
inconvenient, but at least we can find code for this, which is 
compatible for multiple JDK versions
* csgen will probably have to fall back to reflection for most cases, 
which means it will be a lot slower. A bigger change would be to give it 
an indy backend. We would probably still suffer a performance penalty, 
since we then have an additional layer with wrapping in between, that 
cannot be removed by the JIT, as long as the JIT is unable to look 
beyond the indy callsite itself.
* the indy part will have to use "forbidden" API now.
* runtime scripts will go into the unnamed module
* hidden APIs and non-public modifiers will be ignored by Groovy for as 
long as the "forbidden" API works, which keeps current semantics
* most of the groovy runtime makes still heavy use of reflection and 
will probably have to be reworked to use indy everywhere, if on JDK9 
(probably JDK8+).

And I am not happy with this outcome so far.

bye Jochen