RFR: 8361211: C2: Final graph reshaping generates unencodeable klass constants

[Aleksey Shipilev]

On Wed, 30 Jul 2025 23:10:27 GMT, Vladimir Kozlov <kvn at openjdk.org> wrote:

> Do you know why logic in `CmpPNode::Ideal()` did not work? That is what @TobiHartmann pointed before.

I have not been able to track it down. The CTW failure I was chasing was highly intermittent and only reproducible in rare conditions. I suspect the same: the conditions where `CmpPNode::Ideal` run are limited and not guaranteed to fold out _all_ the unencodable constants.

> Would be interesting to track it done because it may cause other issues.

I agree this likely points to a more widespread problem. To be honest, I am pretty horrified that we emit the unencodeable `ConN`-s, and _then_ rely on various node idealization rules to knock them down. So now, if we hold `ConN`, we cannot be sure it would not break down the road! Speaking of nightmarish scenarios, I cannot see, for example, what prevents a particular arch-specific matching rule to assume that `ConN` is encodeable and start doing tricks based on that assumption. 

This PR only handles one limited case in final graph reshaping where I saw this definitely failing. But we also emit these constants as the matter of course during parsing, see the [JDK-8343218](https://bugs.openjdk.org/browse/JDK-8343218) comment for example stack trace. I think we must do a change that puts the abstract/interface encoding optimization behind the feature flag, and we should disable that flag by default, until we are 99.(9)% sure C2 is immune to these issues. A better place to discuss this would be [JDK-8343218](https://bugs.openjdk.org/browse/JDK-8343218). Meanwhile, I would like to plug the leak in final graph reshaping with this PR.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/26559#issuecomment-3139062739