ClassFileTransformer does not apply to anonymous classes

[Rafael Winterhalter]

Hello everybody,

classes that are loaded via Unsafe::defineAnonymousClass are not
transformed by a registered ClassFileTransformer. At the same time, it is
possible to retransform / redefine such an anonymous classes using the
instrumentation API.

Here is a rather confusing bug that I encountered when working with an
internally used Java agent after upgrading a code base to Java 8:

The Java agent attaches at runtime (agentmain). It then registers a
ClassFileTransformer that also applies for retransformation. Afterwards,
all loaded classes that fullfil a given condition are explicitly registered
to be retransformed by the agent. (Doing so, anonymous classes are returned
by Instrumentation::getAllLoadedClasses.) This resulted in the following
behavior of the agent:

a) If the agent was attached after an anonymous class was loaded, the
retransformation would apply to an anonymous class.
b) If the agent was attached "too early", the (non-re-)transformation would
not apply to an anonymous class.

I wonder if it is intentional that a ClassFileTransformer is not applied
when an anonymous class is loaded. Personally, I find this
counter-intuitive, especially when converting anonymous inner classes to
lambda expressions where many users of instrumentation do not forsee the
behavioral change. It also puts a very unforseeable limit to the
instrumentation API. I would therefore like to suggest that
ClassFileTransformers are also applied to anonymous classes when
Unsafe::defineAnonymousClass is called just as when going via
ClassLoader::defineClass.

I can tell that this behavior has not only affected me as I had this
question comming up by multiple users of my open-source code generation
library.

What is your view on this?

Thank you for your feedback!
Best regards, Rafael