From liangchenblue at gmail.com  Fri Jun  6 02:01:02 2025
From: liangchenblue at gmail.com (Chen Liang)
Date: Fri, 6 Jun 2025 10:01:02 +0800
Subject: Long-term maintenance of verifier
Message-ID: <CABe8uE3PKbUfn37O02caoSxiwijS_bOgfT7hAHfSMATnxsr2+g@mail.gmail.com>

Hello,
I have noted that the classfile API's copy of migrated verifier seems to
naturally diverge from the c++ code: for example, JDK-8350029 that
restricts invokespecial to not allow invoking arbitrary interface methods
is not shadowed to the classfile verifier. This problem will only get more
serious once strict fields are added. Meanwhile, people expect
ClassFile.verify to be up-to-date with the runtime verifier.

What should we do to resolve this discrepancy? Should we have a separately
maintained Java-based verifier implementing JVMS 4.10, or should we just
increase our frequency of synchronizing with runtime?

Regards,
Chen Liang
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/classfile-api-dev/attachments/20250606/df3fc75c/attachment.htm>

From adam.sotona at oracle.com  Fri Jun  6 06:49:45 2025
From: adam.sotona at oracle.com (Adam Sotona)
Date: Fri, 6 Jun 2025 06:49:45 +0000
Subject: Long-term maintenance of verifier
In-Reply-To: <CABe8uE3PKbUfn37O02caoSxiwijS_bOgfT7hAHfSMATnxsr2+g@mail.gmail.com>
References: <CABe8uE3PKbUfn37O02caoSxiwijS_bOgfT7hAHfSMATnxsr2+g@mail.gmail.com>
Message-ID: <DS0PR10MB68472863EEFC08D43F00C8DE8C6EA@DS0PR10MB6847.namprd10.prod.outlook.com>

Java copy of the class file verifier played critical role in the Class-File API development.
Now it is important for the Class-File API tests, and it also serves as useful tool to Class-File API users.
Despite its effort to identify all violations of the JVMS 4.10, it does not cover 100%.

There were (and still are) three options how to handle this situation:

  1.  Use the native class file verifier in the Class-File API.
An attempt to implement this option failed at the early stage of the Class-File development. It requires significant implementation changes on the native verifier side or class loading of each involved class.

  2.  A Java copy of the native class file verifier.
That is the current solution, which serves its limited purpose, and it requires synchronization.

  3.  A single Java-based class file verifier implementation.
Complexity of such solution is the highest because it needs to perfectly serve two different purposes. We can start think of the native class file verifier as soon as all the coverage, compatibility, API design, bootstrap, and performance obstacles are resolved.

My suggestion would be to keep the ?easy-to-synchronize? Java copy of the native class file verifier and keep it up to date.
Until a new class file verifier implementation appears, with capability to replace them both. However, that would not be an easy task.

Thanks,
Adam

From: classfile-api-dev <classfile-api-dev-retn at openjdk.org> on behalf of Chen Liang <liangchenblue at gmail.com>
Date: Friday, 6 June 2025 at 4:01
To: classfile-api-dev <classfile-api-dev at openjdk.org>
Subject: Long-term maintenance of verifier
Hello,
I have noted that the classfile API's copy of migrated verifier seems to naturally diverge from the c++ code: for example, JDK-8350029 that restricts invokespecial to not allow invoking arbitrary interface methods is not shadowed to the classfile verifier. This problem will only get more serious once strict fields are added. Meanwhile, people expect ClassFile.verify to be up-to-date with the runtime verifier.

What should we do to resolve this discrepancy? Should we have a separately maintained Java-based verifier implementing JVMS 4.10, or should we just increase our frequency of synchronizing with runtime?

Regards,
Chen Liang
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/classfile-api-dev/attachments/20250606/2ab12970/attachment-0001.htm>

From brian.goetz at oracle.com  Fri Jun  6 15:46:47 2025
From: brian.goetz at oracle.com (Brian Goetz)
Date: Fri, 6 Jun 2025 11:46:47 -0400
Subject: Long-term maintenance of verifier
In-Reply-To: <CABe8uE3PKbUfn37O02caoSxiwijS_bOgfT7hAHfSMATnxsr2+g@mail.gmail.com>
References: <CABe8uE3PKbUfn37O02caoSxiwijS_bOgfT7hAHfSMATnxsr2+g@mail.gmail.com>
Message-ID: <015b6e7b-b5af-47b4-aa1f-28a99b25eece@oracle.com>

We had some discussions about this when the verifier was hand-translated 
from the C++ code.? At the time, some care was taken to ensure that we 
_not_ refactor-as-we-go, to preserve some structural similarity with the 
C++ code, so that deltas in the C++ code could be forward-ported.

We were initially skeptical that duplicating this code would work, but 
it turned out to be quite effective.? Of course, long-term maintenance 
has to be part of the story.

Going forward, I think we are still in the phase where the "keep it 
synchronized" strategy (Adam's #2) can work.? We have to be diligent to 
not make stylistic changes as we go, and keep up with changes in the 
runtime.

Long term, I think there may be something we can do with #1, now that 
Panama is finalized, but that is not enough.? While calling into the C++ 
code is now more practical, but that's not enough; the runtime code is 
designed solely to support on-the-fly analysis during class loading.? 
Some refactoring of the C++ API, undertaken by the runtime team, would 
be needed to allow it to serve multiple masters.

On 6/5/2025 10:01 PM, Chen Liang wrote:
> Hello,
> I have noted that the classfile API's copy of migrated verifier seems 
> to naturally diverge from the c++ code: for example, JDK-8350029 that 
> restricts invokespecial to not allow invoking arbitrary interface 
> methods is not shadowed to the classfile verifier. This problem will 
> only get more serious once strict fields are added. Meanwhile, people 
> expect ClassFile.verify to be up-to-date with the runtime verifier.
>
> What should we do to resolve this discrepancy? Should we have a 
> separately maintained Java-based verifier implementing JVMS 4.10, or 
> should we just increase our frequency of synchronizing with runtime?
>
> Regards,
> Chen Liang
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/classfile-api-dev/attachments/20250606/8a2061b1/attachment.htm>

From liangchenblue at gmail.com  Sat Jun  7 01:31:02 2025
From: liangchenblue at gmail.com (Chen Liang)
Date: Sat, 7 Jun 2025 09:31:02 +0800
Subject: Long-term maintenance of verifier
In-Reply-To: <015b6e7b-b5af-47b4-aa1f-28a99b25eece@oracle.com>
References: <CABe8uE3PKbUfn37O02caoSxiwijS_bOgfT7hAHfSMATnxsr2+g@mail.gmail.com>
 <015b6e7b-b5af-47b4-aa1f-28a99b25eece@oracle.com>
Message-ID: <CABe8uE1vnCEP3jOxYKqCCQjaNyKs0gOnkONjTgFHvxZrN=1ObA@mail.gmail.com>

Thanks for the clarification. I totally agree with this decision of staying
on track of Option 2, as verification in the ClassFile API is mostly a
debug tool instead of a performance sensitive component.

That said, I noted we are a bit out of sync - for example, JDK-8350029 is
applied to runtime recently and it is clearly not in the ClassFile API. On
the mainline, lack of synchronization is not that big a deal as we have
only minor updates; however, project valhalla introduces strict fields,
which makes this synchronization more necessary, especially that otherwise
strict fields and early_larval_frame will be regarded as verification
errors.

We can probably start our updating on mainline so we can help valhalla. We
currently can claim the CF verifier lacks any patch after the initial
6/10/2022 publication in jdk-sandbox, but it is still helpful to know which
commit or which JDK release of verifier.cpp was the copy based off - we can
probably drop this "up-to-date-to-tag/commit" info in our shadow so we can
sync more easily.

Adam, do you know about the exact fork version of the verifier code? Once
we have a point in history, it's quite easy to check the hotspot commits
and see if we need to update our copy accordingly.

Regards, Chen

On Fri, Jun 6, 2025 at 11:46?PM Brian Goetz <brian.goetz at oracle.com> wrote:

> We had some discussions about this when the verifier was hand-translated
> from the C++ code.  At the time, some care was taken to ensure that we
> _not_ refactor-as-we-go, to preserve some structural similarity with the
> C++ code, so that deltas in the C++ code could be forward-ported.
>
> We were initially skeptical that duplicating this code would work, but it
> turned out to be quite effective.  Of course, long-term maintenance has to
> be part of the story.
>
> Going forward, I think we are still in the phase where the "keep it
> synchronized" strategy (Adam's #2) can work.  We have to be diligent to not
> make stylistic changes as we go, and keep up with changes in the runtime.
>
> Long term, I think there may be something we can do with #1, now that
> Panama is finalized, but that is not enough.  While calling into the C++
> code is now more practical, but that's not enough; the runtime code is
> designed solely to support on-the-fly analysis during class loading.  Some
> refactoring of the C++ API, undertaken by the runtime team, would be needed
> to allow it to serve multiple masters.
>
> On 6/5/2025 10:01 PM, Chen Liang wrote:
>
> Hello,
> I have noted that the classfile API's copy of migrated verifier seems to
> naturally diverge from the c++ code: for example, JDK-8350029 that
> restricts invokespecial to not allow invoking arbitrary interface methods
> is not shadowed to the classfile verifier. This problem will only get more
> serious once strict fields are added. Meanwhile, people expect
> ClassFile.verify to be up-to-date with the runtime verifier.
>
> What should we do to resolve this discrepancy? Should we have a separately
> maintained Java-based verifier implementing JVMS 4.10, or should we just
> increase our frequency of synchronizing with runtime?
>
> Regards,
> Chen Liang
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/classfile-api-dev/attachments/20250607/463a92f7/attachment-0001.htm>

From roger.riggs at oracle.com  Mon Jun  9 14:18:33 2025
From: roger.riggs at oracle.com (Roger Riggs)
Date: Mon, 9 Jun 2025 10:18:33 -0400
Subject: Long-term maintenance of verifier
In-Reply-To: <CABe8uE1vnCEP3jOxYKqCCQjaNyKs0gOnkONjTgFHvxZrN=1ObA@mail.gmail.com>
References: <CABe8uE3PKbUfn37O02caoSxiwijS_bOgfT7hAHfSMATnxsr2+g@mail.gmail.com>
 <015b6e7b-b5af-47b4-aa1f-28a99b25eece@oracle.com>
 <CABe8uE1vnCEP3jOxYKqCCQjaNyKs0gOnkONjTgFHvxZrN=1ObA@mail.gmail.com>
Message-ID: <8dd1f9be-2a90-437c-a153-8db58ec8c9db@oracle.com>

HI,

One way to detect getting out of sync, is to structure the test cases 
for the native verifier such that they can be run by the classfile 
verifier.? Especially use for for test cases developed for the latest 
spec changes.

$.02, Roger

On 6/6/25 9:31 PM, Chen Liang wrote:
> Thanks for the clarification. I totally agree with this decision of 
> staying on track of Option 2, as verification in the ClassFile API is 
> mostly a debug tool instead of a performance sensitive component.
>
> That said, I noted we are a bit out of sync - for example, JDK-8350029 
> is applied to runtime recently and it is clearly not in the ClassFile 
> API. On the mainline, lack of synchronization is not that big a deal 
> as we have only minor updates; however, project valhalla introduces 
> strict fields, which makes this synchronization more necessary, 
> especially that otherwise strict fields and early_larval_frame will be 
> regarded as verification errors.
>
> We can probably start our updating on mainline so we can help 
> valhalla. We currently can claim the CF verifier lacks any patch after 
> the initial 6/10/2022 publication in jdk-sandbox, but it is still 
> helpful to know which commit or which JDK release of verifier.cpp was 
> the copy based off - we can probably drop this 
> "up-to-date-to-tag/commit" info in our shadow so we can sync more easily.
>
> Adam, do you know about the exact fork version of the verifier code? 
> Once we have a point in history, it's quite easy to check the hotspot 
> commits and see if we need to update our copy accordingly.
>
> Regards, Chen
>
> On Fri, Jun 6, 2025 at 11:46?PM Brian Goetz <brian.goetz at oracle.com> 
> wrote:
>
>     We had some discussions about this when the verifier was
>     hand-translated from the C++ code.? At the time, some care was
>     taken to ensure that we _not_ refactor-as-we-go, to preserve some
>     structural similarity with the C++ code, so that deltas in the C++
>     code could be forward-ported.
>
>     We were initially skeptical that duplicating this code would work,
>     but it turned out to be quite effective.? Of course, long-term
>     maintenance has to be part of the story.
>
>     Going forward, I think we are still in the phase where the "keep
>     it synchronized" strategy (Adam's #2) can work.? We have to be
>     diligent to not make stylistic changes as we go, and keep up with
>     changes in the runtime.
>
>     Long term, I think there may be something we can do with #1, now
>     that Panama is finalized, but that is not enough. While calling
>     into the C++ code is now more practical, but that's not enough;
>     the runtime code is designed solely to support on-the-fly analysis
>     during class loading.? Some refactoring of the C++ API, undertaken
>     by the runtime team, would be needed to allow it to serve multiple
>     masters.
>
>     On 6/5/2025 10:01 PM, Chen Liang wrote:
>>     Hello,
>>     I have noted that the classfile API's copy of migrated verifier
>>     seems to naturally diverge from the c++ code: for example,
>>     JDK-8350029 that restricts invokespecial to not allow invoking
>>     arbitrary interface methods is not shadowed to the classfile
>>     verifier. This problem will only get more serious once strict
>>     fields are added. Meanwhile, people expect ClassFile.verify to be
>>     up-to-date with the runtime verifier.
>>
>>     What should we do to resolve this discrepancy? Should we have a
>>     separately maintained Java-based verifier implementing JVMS 4.10,
>>     or should we just increase our frequency of synchronizing with
>>     runtime?
>>
>>     Regards,
>>     Chen Liang
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/classfile-api-dev/attachments/20250609/a8ab272d/attachment.htm>