RFR: 8297727: Forcing LF interpretation lead to StackOverflowError in reflection code

[Chen Liang]

On Wed, 30 Apr 2025 13:46:57 GMT, Jorn Vernee <jvernee at openjdk.org> wrote:

>> When LambdaForms are interpreted, so are field lambda forms. When this happens, we may get into an infinite recursion due to field lambda forms using `MethodHandleStatics.UNSAFE` via field lambda form.
>> 
>> I think the best solution here is to always eagerly compile field DMH forms - we already do so for all method forms, and there is a finite number of field forms, mostly pregenerated. The reference with cast form is erroneously missed, and it is actually the culprit of this issue. Will pregenerate that in another patch.
>
> src/java.base/share/classes/java/lang/invoke/DirectMethodHandle.java line 217:
> 
>> 215:         }
>> 216:         LambdaForm lform = preparedLambdaForm(mtype, which);
>> 217:         maybeCompile(lform, m);
> 
> Ok, looks like this can be dropped because we always call `compileToBytecode` in `makePreparedLambdaForm`?

Indeed, maybeCompile is actually weaker as it does not replace interpreter vmentry

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/24835#discussion_r2068730862