Classes on the stack trace (was: getElementClass/StackTraceElement, was: @CallerSensitive public API, was: sun.reflect.Reflection.getCallerClass)

[Jochen Theodorou]

Am 30.07.2013 16:16, schrieb Peter Levart:
>
> On 07/30/2013 03:19 PM, Jochen Theodorou wrote:
>> Am 30.07.2013 14:17, schrieb Peter Levart:
>> [...]
>>> So what would give Groovy or other language runtimes headaches when all
>>> there was was a parameter-less getCallerClass() API? Aren't the
>>> intermediate frames inserted by those runtimes controlled by the
>>> runtimes? Couldn't the "surface" runtime-inserted methods capture the
>>> caller and pass it down? I guess the problem is supporting calling the
>>> caller-sensitive methods like Class.forName(String) and such which don't
>>> have the overloaded variant taking caller Class or ClassLoader as an
>>> argument...
>> Speaking for Groovy...
>> those intermediate frames are runtime controlled, yes, but passing down
>> the caller class is exactly the problem. Imagine I would suggest that
>> each and every method definition in Java automatically gets an
>> additional parameter for the caller class, just to have access to it
>> inside the method. You would not accept that for Java, would you? And so
>> we cannot accept that for Groovy if we want to keep integration with
>> Java...
>
> Are you talking about internal Groovy implementation (the
> runtime-inserted methods) or the publicly visible API?

that's the problem, it is a mix, some internal, other not. We are going 
to change that in Groovy 3

> One solution for
> internal implementation of Groovy could be (speaking by heart since I
> don't know the internals of Groovy) for the "surface" public API method
> which doesn't have to have the special caller parameter, to capture the
> caller with getCallerClass() parameterless API (possibly enclosed with a
> quick check confirming that it might actually be needed) and bind it to
> a ThreadLocal, then use this ThreadLocal down at the end...

confirming that it might actually be needed is a problem. In the old 
fallback path we don't know what we call until after we are deep in 
runtime code, and there it is too late. In the other paths we could mark 
those methods in a @CallerSensitive style and do it in that case only.

>> and the good integration with Java is one of the key points of
>> Groovy. Even if we make something like that @CallerSensitive and add the
>> parameter only in those cases, we break being able to override methods.
>
> I guess I don't know every Groovy need to obtain the caller class. I
> thought the problem was to support calling caller-sensitive methods in
> Java API (like Class.forName(String)) from within Groovy code, where
> there are runtime-inserted frames between the "call-site" and the target
> method. Are there any other needs?

ok, there is a misunderstanding...

if we call a Java implemented method from Groovy, which is using 
getCallerClass() it may or may not work. In general this does not work 
and our problem is not about that at all. With the change to let 
getCallerClass() ignore some reflective frames it will work actually 
better as long as we use our custom callsite caching implementation, it 
will not work if indy is used or the fallback path.

To be able to call a method Class#forName(String), we need to "replace" 
it with an implementation of our own, which we do with an approach 
similar to extension methods (only that ours can hide existing 
implementation methods for groovy). And in there we need to get to the 
caller class

Our problem though is @Grab which is an annotation to add elements to 
the classpath while running a script.

>> Plus, before Groovy3 is not done we have to support several call paths.
>> And the oldest one, which is still a fallback, does not support
>> transporting the caller class through the runtime layers at all.
>> Changing here is a breaking change.
>
> Could you describe those call-paths? Examples of Groovy code and to what
> it gets translated (equivalent Java code at call site) with a brief
> description of what each intermediate layer (between the call-site and
> the target method) does and at which point the caller class is extracted...

the code generated at the call site depends on several factors 
actually... The call site code itself is usually not very informative

I start with Groovy 1.0, since that is basically the fallback path. Here 
this.foo() translates more or less to
ScriptBytecodeAdapter.invokeMethod0(staticCallerClass, this,"foo")
which basically does this.getMetaClass().invokeMethod(staticCallerClass, 
this, "foo"). The problem is that the meta class might be user supplied 
and the code executed in invokeMethod as well. The invocation is then 
finally done by reflection. That means we have frames from 
ScriptBytecodeAdapter, from the meta class, as well as maybe frames from 
a custom meta class and reflection frames. At the level of 
ScriptBytecodeAdapter there is a means of transporting the caller class, 
but that is the static one. Once there is a subclass, this information 
is different from what is needed here and it cannot simply be exchanged. 
Even if the bytecode adapter is changed, we cannot change the public API 
for MetaClass#invokeMethod now. And then the information would be lost.

In later versions of Groovy (since 1.6) we introduced a custom call site 
caching technique, which uses runtime generated classes to create a 
helper class per call site and is then used for invocation. At the 
callsite we basically have something like callsiteArray[i].invoke(..). 
Here again the staticCallerClass can be found. In this version we are 
able to "get" the method we want to invoke, before invoking it 
(bypassing MetaClass#invokeMethod). But to be able to get the method, 
certain conditions have to be met (like no user supplied meta class). If 
they are not met, then we do basically the same path as in 1.0, only 
that we don't use ScriptBytecodeAdapter. Instead We use our CallSite 
class as entrance point, which then makes the call to the meta class. In 
the "efficent" case we have now frames from the callsite handling code 
between the callsite and the target method only. This includes 
reflection in the first instantiation, later the generated class is used 
so it reduces to two frames of which one is the Callsite entrance point, 
the other a frame form the generated method. In the fallback case we 
have frames from the callsite handling code, plus meta class code, plus 
reflection of course. Again the fallback case prevents us from 
transporting the caller information to the target method. If we ignore 
the fallback case, then we could here maybe use the Threadlocal 
information. It will require a new callsite interface for the bytecode 
though, meaning this code will not work for precompiled grovvy of older 
version, excluding from getting into Groovy 2.1.x, since it would be a 
breaking change. The earliest version for that would be Groovy 2.2.0, 
which is almost in RC now. Effectively it would mean we would have to do 
a 2.3.0 very soon after most probably.

In Groovy 2 we added an indy implementation, which replaces the callsite 
caching code.  At the callsite we have here basically invokedynamic 
"foo" with IndyInterface#bootstrap. bootstrap will first introduce a 
target for IndyInterface#selectMethod, since I need the runtime types 
instead of the static ones. The static caller class information is here 
part of the bootstrap method as Lookup object, added by invokedynamic 
itself. After selectMethod is done we have an initial invocation using 
invokeExact and later invocations by the handle stored in the callsite. 
Of course the same conditions as for the callsite caching above have to 
be met, meaning the fallback path might appear. That makes initially one 
IndyInterface frame, then invokedynamic and lambda related frames, then 
optionally the traget method, or in the fallback case the meta class 
frames plus reflection


bye Jochen

-- 
Jochen "blackdrag" Theodorou - Groovy Project Tech Lead
blog: http://blackdragsview.blogspot.com/
german groovy discussion newsgroup: de.comp.lang.misc
For Groovy programming sources visit http://groovy-lang.org