RFR: 8361211: C2: Final graph reshaping generates unencodeable klass constants

[Tobias Hartmann]

On Wed, 30 Jul 2025 16:20:43 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

> See the bug for more investigation. I have tried to come up with an isolated test, but failed. So I am doing this change somewhat blindly, without a clear regression test. The investigation on the CTW points directly to this code, and I believe we should be more conservative in final graph reshaping. [JDK-8343206](https://bugs.openjdk.org/browse/JDK-8343206) added the assert for `ConNKlass`, which somehow does not trigger. I think it is safe to bail out of this transformation.
> 
> Also, this only plugs this particular leak. I think we should really be disabling the abstract/interface encoding optimization until C2 does not expose itself to this issue on more paths. There is [JDK-8343218](https://bugs.openjdk.org/browse/JDK-8343218) that we can re-open.
> 
> Additional testing:
>  - [x] Linux x86_64 server fastdebug, a rare CTW failure does not reproduce anymore
>  - [x] Linux x86_64 server fastdebug, `tier1`
>  - [x] Linux x86_64 server fastdebug, `all`
>  - [x] Linux AArch64 server fastdebug, `all`

> Friendly ping :)

Sorry for the delay. Public holiday, traveling and JVMLS now.

> [JDK-8343206](https://bugs.openjdk.org/browse/JDK-8343206) added the assert for ConNKlass, which somehow does not trigger

That's probably because the ConNKlass is not added on the `nstack` and therefore not visited by the final graph reshaping code anymore.

-------------

PR Review: https://git.openjdk.org/jdk/pull/26559#pullrequestreview-3085897801