ClassFileTransformer does not apply to anonymous classes

Fri Jan 22 16:19:12 UTC 2016

Hi Vladimir,

thank you for your response. While I completely understand your view
from a VM implementor's point of view, as a practicioner I would
recommend against it. Not being able to instrument lambda expressions
puts a severe limitation onto using the instrumentation API
alltogether. For example, a monitoring application that promises to
record all invocations of a method of a certain interface that only
works 95% of the time is not 5% less usefull but might no longer be
useful at all. People have build large applications based on the
assumption that all user application code can be instrumented and such
a regression would hurt them. For example, until today, over 30 people
that use my code generation framework reached out to me and reported
the reported behavior as a bug in my library and people are only
starting to migrate their applications to Java 8. The outcome is
simply not intuitive as using a lambda expression over an anonyous
inner class should not change an application's behavior.

The currently used workaround that people use is to instrument the
LambdaMetaFactory class itself to apply the transformer manually when
it is being created. This solution is spreading as a standard approach
and I am sure that forbidding a redefinition alltogether would only
inspire to other workarrounds for being able to migrate running code
to the next Java version.

Furthermore, I do not think that not being able to patch constant pool
indices in the generated code introduces any problems in practice.
Most real-world instrumentation only appends new constant pool entries
without interfering with the existant pool. Rather, I would like to
see an exception being raised if one attempts to change the original
constant pool without me being able to consider this from a technical
perspective. I think this would serve to be sufficient for most
people.

I really hope that there is a way to allow for patching anonymously
loaded classes even without exposing the constant pool patches.
So far, thank you for looking into this. I am sure this a more complex
matter than what I am able to comprehend. I appreciate that you are
taking the time to consider my opinion!

Best regards, Rafael

2016-01-22 16:42 GMT+01:00 Vladimir Ivanov <vladimir.x.ivanov at oracle.com>:
> Rafael,
>
> What you are seeing are just consequences. My impression is that VM
> anonymous class redefinition/retransformation works mostly by accident.
>
> The real problem is that current API (both JVMTI and java.lang.instrument)
> doesn't serve for them well.
>
> When VM anonymous class is defined, a user provides both bytecode and CP
> patches:
>     public native Class<?> defineAnonymousClass(Class<?> hostClass, byte[]
> data, Object[] cpPatches);
>
> Unless there's a way to adjust CP patches along with the bytecode, patching
> bytecode is very fragile (e.g., no way to add new patches, should keep
> patched CP entry indexes intact).
>
> The root cause is Unsafe.defineAnonymousClass is part of
> implementation-specific private API, so no way to expose it a public APIs
> (special entry points in JVMTI or java.lang.instrument).
>
> Also, though the concept of VM anonymous class was a step in the right
> direction, it isn't good enough to be standardized. The ultimate goal is to
> get lightweight code loading mechanism, but VM anonymous classes are still
> loaded with class semantics.
>
> So, I don't think we can do much w.r.t. VM anonymous classes. I'd prefer JVM
> to skip agent notifications or completely forbidding VM anonymous class
> retransformation/redefinition, but I haven't thought through compatibility
> consequences.
>
> Best regards,
> Vladimir Ivanov
>
>
> On 1/22/16 5:04 PM, Rafael Winterhalter wrote:
>>
>> Hi Coleen, thanks for looking into this. My original mail was the
>> following:
>>
>> Hello everybody,
>>
>> classes that are loaded via Unsafe::defineAnonymousClass are not
>> transformed by a registered ClassFileTransformer. At the same time, it
>> is possible to retransform / redefine such an anonymous classes using
>> the instrumentation API.
>>
>> Here is a rather confusing bug that I encountered when working with an
>> internally used Java agent after upgrading a code base to Java 8:
>>
>> The Java agent attaches at runtime (agentmain). It then registers a
>> ClassFileTransformer that also applies for retransformation.
>> Afterwards, all loaded classes that fullfil a given condition are
>> explicitly registered to be retransformed by the agent. (Doing so,
>> anonymous classes are returned by
>> Instrumentation::getAllLoadedClasses.) This resulted in the following
>> behavior of the agent:
>>
>> a) If the agent was attached after an anonymous class was loaded, the
>> retransformation would apply to an anonymous class.
>> b) If the agent was attached "too early", the (non-re-)transformation
>> would not apply to an anonymous class.
>>
>> I wonder if it is intentional that a ClassFileTransformer is not
>> applied when an anonymous class is loaded. Personally, I find this
>> counter-intuitive, especially when converting anonymous inner classes
>> to lambda expressions where many users of instrumentation do not
>> forsee the behavioral change. It also puts a very unforseeable limit
>> to the instrumentation API. I would therefore like to suggest that
>> ClassFileTransformers are also applied to anonymous classes when
>> Unsafe::defineAnonymousClass is called just as when going via
>> ClassLoader::defineClass.
>>
>> I can tell that this behavior has not only affected me as I had this
>> question comming up by multiple users of my open-source code
>> generation library.
>>
>> What is your view on this?
>>
>> Thank you for your feedback!
>> Best regards, Rafael
>>
>> 2016-01-22 15:00 GMT+01:00 Coleen Phillimore
>> <coleen.phillimore at oracle.com>:
>>>
>>>
>>>
>>> On 1/22/16 4:11 AM, Andrew Dinn wrote:
>>>>
>>>>
>>>> On 21/01/16 22:14, Rafael Winterhalter wrote:
>>>>>
>>>>>
>>>>> Hi Andrew,
>>>>> if there is any update on the matter, I would of course love to hear
>>>>> about it.
>>>>> Unfortunately, I never received an answer to my question, but maybe I
>>>>> picked the wrong list.
>>>>> Thank you for your support on this issue!
>>>>
>>>>
>>>> I'm pretty sure this is the right list. But I don't know who cold answer
>>>> the question. I know that Coleen Phillmore (added explicitly to CC)
>>>> often deals with ClassFileTransformer issues. Coleen, can you answer?
>>>
>>>
>>>
>>> Can you send the question again?  I didn't see it.  I also mostly look at
>>> JVM code and rarely deal with the Java side.
>>>
>>> I'm also adding Dan, Serguei and Markus.
>>>
>>> Thanks,
>>> Coleen
>>>
>>>>
>>>> regards,
>>>>
>>>>
>>>> Andrew Dinn
>>>> -----------
>>>> Senior Principal Software Engineer
>>>> Red Hat UK Ltd
>>>> Registered in UK and Wales under Company Registration No. 3798903
>>>> Directors: Michael Cunningham (US), Michael O'Neill (Ireland), Paul
>>>> Argiry (US)
>>>
>>>
>>>
>