RFR: 8361211: C2: Final graph reshaping generates unencodeable klass constants [v2]

[Quan Anh Mai]

On Tue, 5 Aug 2025 10:25:45 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> See the bug for more investigation. I have tried to come up with an isolated test, but failed. So I am doing this change somewhat blindly, without a clear regression test. The investigation on the CTW points directly to this code, and I believe we should be more conservative in final graph reshaping. [JDK-8343206](https://bugs.openjdk.org/browse/JDK-8343206) added the assert for `ConNKlass`, which somehow does not trigger. I think it is safe to bail out of this transformation.
>> 
>> Also, this only plugs this particular leak. I think we should really be disabling the abstract/interface encoding optimization until C2 does not expose itself to this issue on more paths. There is [JDK-8343218](https://bugs.openjdk.org/browse/JDK-8343218) that we can re-open.
>> 
>> Additional testing:
>>  - [x] Linux x86_64 server fastdebug, a rare CTW failure does not reproduce anymore
>>  - [x] Linux x86_64 server fastdebug, `tier1`
>>  - [x] Linux x86_64 server fastdebug, `all`
>>  - [x] Linux AArch64 server fastdebug, `all`
>
> Aleksey Shipilev has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains three additional commits since the last revision:
> 
>  - Assert and bailout for ConP -> EncodeP path
>  - Merge branch 'master' into JDK-8361211-c2-encodeable
>  - Fix

Marked as reviewed by qamai (Committer).

-------------

PR Review: https://git.openjdk.org/jdk/pull/26559#pullrequestreview-3090283443