Does API for transformation of class files in Class-FIle API solves the same problem as code generationg annotation processors?

Wed Sep 18 10:26:51 UTC 2024

>
> It does not control, nor can it control, where the byte data comes from or
> where the byte data goes.

That's true. Although, why I am saying that one of the Class-File API goals
is to allow semantic modifications, is because the authors of API
specifically said that when asked about use cases. The talk that
particularly motivated me to write the letter was Brian Goetz talk on Java
23 launch stream, where he said something along the lines "add a few
bytecodes at the start of the method". So this is not an uncontrollable
side effect, but rather an intentional way to use API.

If you consider bytecode modification unsafe, you can use ClassFile::verify
> to ensure your generated bytecode passes verification.

I am not that familiar with API, but I'm sure it has its limits. I'm sure
it is capable of verifying classfiel structure, but I highly doubt that it
can enforce things like type safety, especially with generics, verifying
pointers to symbols are correct etc. WOuld be glad to be mistaken though.

For semantic change: Annotation processors change the semantics of the
> language and start from an invalid language semantic; meanwhile, classfile
> transformation start valid and end with a valid class file format. I think
> this is the principal difference here.

That is true if the assumption that annotations can't change semantics of
code like keywords do. That`s why I am wondering if it still stays the
same, since class-file api seems to legalize these changes. Yes, class
files are valid on start and valid in the end, but that doesn't change the
fact that semantics is changed. Moreover, some annotation processors also
start with valid semantics and end with valid one, let's say
lombok`s @Locked, @EqualsAndHashCode etc. Other annotations like @Getter
and @Setter are not more "illegal" at the start than any processor that
generates additional classes, since symbols are unresolved in both of these
cases. Annotation processors are still obliged to pass syntax checks, they
can't invent new language constructs, keywords or statements. So the
annotation processor is not obliged to start with invalid semantics.
Moreover, if annotations are now effectively allowed to change code
semantics, then the "code semantics" itself could be stretched
indefinitely. If annotation of type guarantees presence of method,
developers knows about it, and annotation is allowed to alter type`s
members list, then semantics seems pretty valid to me, at least as valid as
if we were to refer to some annotations like mapstruct`s @Mapper, which is
completely build on public APIs. Having all that, I, personally, don't see
how processors are more harmful then bytecode transformations.

>From now on, this is just my thoughts on annotation processing in the
ecosystem.

I, personally, don't see any particular harm in processors like Lombok,
that are used to eliminate boilerplate code. Anyone who ever wrote code
using it knows that it dramatically enhances both writability and
readability of code. I think the fact that lombok is even present in spring
boot starters speaks to the popularity of the library a lot. Of course,
there are much more invasive ones, that I am not a fan of honestly.
Although, what I think, is that if there were some "legal" extent to which
annotation processors could invade in existing code, say wrapping method
bodies and generating new members for existing types and a few others,
which would cover 99% of sane use cases, this would encourage people to use
and write "sane" annotation processors that are proven to improve
productivity and discourage use of "crazy" ones due to stability of premier
ones.

On Wed, Sep 18, 2024 at 2:02 AM Chen Liang <chen.l.liang at oracle.com> wrote:

> Hi Olex,
> ClassFile API provides a general-purpose API to interact with byte data
> for the class file format. It does not control, nor can it control, where
> the byte data comes from or where the byte data goes. So, the API doesn't
> know if you are upgrading old formats, such as migrating to value classes
> or removing jsr instructions, or constant-folding expressions to dynamic
> constants, or to add dangerous hacks to the class file. And there is no way
> to control it for the API. The only way you can safeguard is to ensure you
> disable dynamic agents and use libraries that you trust. Our current
> recommendation is to use ClassFile API to pre-process (similar to
> annotation processing in compilation), such as putting all candidate
> classes in a flat directory; run-time bytecode loading (such as minecraft
> mods) is much harder to optimize (such as for Leyden) and difficult to
> detect errors.
>
> If you consider bytecode modification unsafe, you can use
> ClassFile::verify to ensure your generated bytecode passes verification.
> This is not a prerequisite to bytecode generation as this process is quite
> slow, but you can always run it in your program or toggle it with a
> property. StackMapTable attribute is quite a good tool to ensure the type
> safety within Java's class file format.
>
> For semantic change: Annotation processors change the semantics of the
> language and start from an invalid language semantic; meanwhile, classfile
> transformation start valid and end with a valid class file format. I think
> this is the principal difference here.
>
> Regards, Chen
> ------------------------------
> *From:* core-libs-dev <core-libs-dev-retn at openjdk.org> on behalf of
> Olexandr Rotan <rotanolexandr842 at gmail.com>
> *Sent:* Tuesday, September 17, 2024 5:49 PM
> *To:* Bernd <ecki at zusammenkunft.net>
> *Cc:* core-libs-dev at openjdk.org <core-libs-dev at openjdk.org>
> *Subject:* Re: Does API for transformation of class files in Class-FIle
> API solves the same problem as code generationg annotation processors?
>
>
> Sorry for followup letter, just mentioned that I wrote that annotations
> CAN change the semantics, while I meant CANNOT. With this typo letter makes
> little to none sense, so correction is important
>
> On Tue, Sep 17, 2024, 22:51 Olexandr Rotan <rotanolexandr842 at gmail.com>
> wrote:
>
> When I said that bytecode modification is unsafe, I was mainly
> regarding type safety. Compiler does a load of work to not let through
> invalid code, and processors that modify ast still have to pass symbol
> resolution checks, type checks and many other ones, while
> bytecode modification basically has no safety mechanisms other then runtime
> failure (and it's good if it fails, there may be just silent errors). In
> this sense, I would regard AST modification as much more preferable option
> if there was a public API for it
>
> On Tue, Sep 17, 2024 at 10:41 PM Bernd <ecki at zusammenkunft.net> wrote:
>
> Hello,
>
> To me bytecode is a standardized interface and creating or modifying
> bytecode is legal and not inherently unsafe.
>
> If annotation processor authors want to use bytecode modifications they
> now get an additional tool for that but it doesn’t change the general usage.
>
> In the end users have to know if they want to use intransparent magic like
> Lombok or Pointcuts or even worse runtime agents. And this affects
> selecting platforms who need it or not. (And don’t forget the ecosystem is
> much bigger than Java language alone)
>
> Gruß
> Bernd
> --
> https://bernd.eckenfels.net
>
> ------------------------------
> *From:* core-libs-dev <core-libs-dev-retn at openjdk.org> on behalf of
> Olexandr Rotan <rotanolexandr842 at gmail.com>
> *Sent:* Tuesday, September 17, 2024 9:16 PM
> *To:* core-libs-dev at openjdk.org <core-libs-dev at openjdk.org>
> *Subject:* Does API for transformation of class files in Class-FIle API
> solves the same problem as code generationg annotation processors?
>
> Hello there. I am writing to address the overlap between class file
> transformation using the ClassFile API and the work done by code-generating
> annotation processors, and whether they ultimately solve the same problem.
>
> Annotations such as @Async and @Transactional in popular frameworks are
> good examples of code semantics being modified or extended at runtime,
> primarily through proxying mechanisms. In these cases, bytecode
> transformation is employed under the hood, as the annotations imply
> asynchronous execution or transactional boundaries.
>
> The general implication that I am aware of about annotations is that they
> can change code semantics. While I am not a fan of this, I can understand
> people that once set this rule. And so Class-File API exposing
> transformation API for class-files seems like legalizing the bypass of this
> rule. I, personally, am a fan of steps in this direction, since I like to
> write code generators that generate code for existing classes for fun from
> time to time, but that doesnt change my assumption that Class-File API
> somehow contradicts general rules applied to annotations by JDK, and it
> always seems kind of odd to me when some of API authors speaks so calmly
> about on-flight transformations of bytecode while even AST transformations
> in compile time aren't supported by JDK.
>
> Moreover, Class-File API seems like more than just an alternative to
> generating processors, but rather a weapon of mass destruction compared to
> later. Changes to bytecode are always unsafe, and their versatility makes
> them much more invasive.
>
> To me, personally, after this API is introduced, addressing long standing
> questions of libraries like lombok and more exotic ones like manifold seems
> to be an organic step. In my opinion, if even bytecode transformations are
> now legal, then much more safe AST transformations (which are safer because
> the compiler wont let through invalid AST, while bytecode transformations
> could cause silent errors) should also be addressed. SInce JDK historically
> mostly focused on code readability, many libraries picked up responsibility
> of easing writability, and "legalizing" them, in my opinion, would be a
> giant step in the right direction.
>
> So what am I missing here? Or is it really as it seems, that now that
> bytecode transformations are in public APIs, making invasive changes in
> compiled code is basically legalized? Would really appreciate it if someone
> could spare some time to make things clear for me.
>
> Best regards
>
> --
> https://bernd.eckenfels.net
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/core-libs-dev/attachments/20240918/833de0ae/attachment-0001.htm>