Does API for transformation of class files in Class-FIle API solves the same problem as code generationg annotation processors?

[Chen Liang]

Hi Olex,
ClassFile API provides a general-purpose API to interact with byte data for the class file format. It does not control, nor can it control, where the byte data comes from or where the byte data goes. So, the API doesn't know if you are upgrading old formats, such as migrating to value classes or removing jsr instructions, or constant-folding expressions to dynamic constants, or to add dangerous hacks to the class file. And there is no way to control it for the API. The only way you can safeguard is to ensure you disable dynamic agents and use libraries that you trust. Our current recommendation is to use ClassFile API to pre-process (similar to annotation processing in compilation), such as putting all candidate classes in a flat directory; run-time bytecode loading (such as minecraft mods) is much harder to optimize (such as for Leyden) and difficult to detect errors.

If you consider bytecode modification unsafe, you can use ClassFile::verify to ensure your generated bytecode passes verification. This is not a prerequisite to bytecode generation as this process is quite slow, but you can always run it in your program or toggle it with a property. StackMapTable attribute is quite a good tool to ensure the type safety within Java's class file format.

For semantic change: Annotation processors change the semantics of the language and start from an invalid language semantic; meanwhile, classfile transformation start valid and end with a valid class file format. I think this is the principal difference here.

Regards, Chen
________________________________
From: core-libs-dev <core-libs-dev-retn at openjdk.org> on behalf of Olexandr Rotan <rotanolexandr842 at gmail.com>
Sent: Tuesday, September 17, 2024 5:49 PM
To: Bernd <ecki at zusammenkunft.net>
Cc: core-libs-dev at openjdk.org <core-libs-dev at openjdk.org>
Subject: Re: Does API for transformation of class files in Class-FIle API solves the same problem as code generationg annotation processors?


Sorry for followup letter, just mentioned that I wrote that annotations CAN change the semantics, while I meant CANNOT. With this typo letter makes little to none sense, so correction is important

On Tue, Sep 17, 2024, 22:51 Olexandr Rotan <rotanolexandr842 at gmail.com<mailto:rotanolexandr842 at gmail.com>> wrote:
When I said that bytecode modification is unsafe, I was mainly regarding type safety. Compiler does a load of work to not let through invalid code, and processors that modify ast still have to pass symbol resolution checks, type checks and many other ones, while bytecode modification basically has no safety mechanisms other then runtime failure (and it's good if it fails, there may be just silent errors). In this sense, I would regard AST modification as much more preferable option if there was a public API for it

On Tue, Sep 17, 2024 at 10:41 PM Bernd <ecki at zusammenkunft.net<mailto:ecki at zusammenkunft.net>> wrote:
Hello,

To me bytecode is a standardized interface and creating or modifying bytecode is legal and not inherently unsafe.

If annotation processor authors want to use bytecode modifications they now get an additional tool for that but it doesn’t change the general usage.

In the end users have to know if they want to use intransparent magic like Lombok or Pointcuts or even worse runtime agents. And this affects selecting platforms who need it or not. (And don’t forget the ecosystem is much bigger than Java language alone)

Gruß
Bernd
--
https://bernd.eckenfels.net

________________________________
From: core-libs-dev <core-libs-dev-retn at openjdk.org<mailto:core-libs-dev-retn at openjdk.org>> on behalf of Olexandr Rotan <rotanolexandr842 at gmail.com<mailto:rotanolexandr842 at gmail.com>>
Sent: Tuesday, September 17, 2024 9:16 PM
To: core-libs-dev at openjdk.org<mailto:core-libs-dev at openjdk.org> <core-libs-dev at openjdk.org<mailto:core-libs-dev at openjdk.org>>
Subject: Does API for transformation of class files in Class-FIle API solves the same problem as code generationg annotation processors?

Hello there. I am writing to address the overlap between class file transformation using the ClassFile API and the work done by code-generating annotation processors, and whether they ultimately solve the same problem.

Annotations such as @Async and @Transactional in popular frameworks are good examples of code semantics being modified or extended at runtime, primarily through proxying mechanisms. In these cases, bytecode transformation is employed under the hood, as the annotations imply asynchronous execution or transactional boundaries.

The general implication that I am aware of about annotations is that they can change code semantics. While I am not a fan of this, I can understand people that once set this rule. And so Class-File API exposing transformation API for class-files seems like legalizing the bypass of this rule. I, personally, am a fan of steps in this direction, since I like to write code generators that generate code for existing classes for fun from time to time, but that doesnt change my assumption that Class-File API somehow contradicts general rules applied to annotations by JDK, and it always seems kind of odd to me when some of API authors speaks so calmly about on-flight transformations of bytecode while even AST transformations in compile time aren't supported by JDK.

Moreover, Class-File API seems like more than just an alternative to generating processors, but rather a weapon of mass destruction compared to later. Changes to bytecode are always unsafe, and their versatility makes them much more invasive.

To me, personally, after this API is introduced, addressing long standing questions of libraries like lombok and more exotic ones like manifold seems to be an organic step. In my opinion, if even bytecode transformations are now legal, then much more safe AST transformations (which are safer because the compiler wont let through invalid AST, while bytecode transformations could cause silent errors) should also be addressed. SInce JDK historically mostly focused on code readability, many libraries picked up responsibility of easing writability, and "legalizing" them, in my opinion, would be a giant step in the right direction.

So what am I missing here? Or is it really as it seems, that now that bytecode transformations are in public APIs, making invasive changes in compiled code is basically legalized? Would really appreciate it if someone could spare some time to make things clear for me.

Best regards

--
https://bernd.eckenfels.net
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/core-libs-dev/attachments/20240917/74ac8d61/attachment-0001.htm>