RFR: 8322726: C2: Unloaded signature class kills argument value

[Vladimir Ivanov]

On Mon, 29 Apr 2024 22:01:37 GMT, John R Rose <jrose at openjdk.org> wrote:

>> For MethodHandle linkers all arguments are casted to signature classes when target method is known.
>> 
>> It causes problems when target method signature contains unloaded classes: when loaded class meets unloaded class it turns into a TOP. It effectively kills argument values which correspond to unloaded signature types.
>> 
>> Proposed fix avoids casts when signature class is unloaded. 
>> 
>> Testing: hs-tier1 - hs-tier4
>
> When a parameter happens to be typed as an unloaded class, the call site can be can be compiled and survive a long time in optimized form, as long as only nulls as passed.  This is because (a) methods do not necessarily issue casts to their arguments, and (b) even if a `checkcast` instruction is issued, it short-circuits on null, without even trying to resolve the `checkcast` class.  C2 supports this corner case, in part, using the `assert_null` IR generation option, which says “as long as this value is null, keep it, otherwise recompile”.
> 
> If we want to emulate this for MHs, we need to ensure that null short-circuits, even if non-null values must somehow cope with the unloaded class.
> 
> (It’s not too hard, but requires some change to configuration, since the non-null value is a witness that the class has been loaded, so the existing code shape, which assumes there is no resolution for the class, is out of date.  BTW a return type can also be an unloaded class, with similar considerations.)
> 
> So ,what do we do on the case of a null value of an unloaded class parameter (or return value) when a call instruction is being emulated by a MH?  And, does this fix change that policy?

@rose00 in this particular case, the problem arises for locally not-yet-loaded classes. Moreover, "truly" unloaded signature classes are not possible for MethodHandles, because their signatures are reified as MethodTypes.  So, asserting null is too strong.

The fix aligns the behavior with bytecode invoke instructions where an unloaded signature class blocks inlining, but no additional checks are issued.

I have a follow-up enhancement to improve behavior for MethodHandles and completely eliminate the case of locally unloaded classes. For bytecode invoke instructions, JVM installs loader constraints for signature classes. We don't do that for MethodHandles. Instead, signature classes are eagerly loaded. But it is omitted for `java.*` classes (as an optimization) and those are the only cases which cause the problem.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/18973#issuecomment-2087339051