RFR: 8311500: StackWalker.getCallerClass() can throw if invoked reflectively [v2]

[Mandy Chung]

On Wed, 5 Jul 2023 17:25:24 GMT, Volker Simonis <simonis at openjdk.org> wrote:

>> As the included jtreg test demonstrates, `StackWalker.getCallerClass()` can throw an `UnsupportedOperationException` if called reflectively. Currently this only happens if we invoke `StackWalker.getCallerClass()` recursively reflectively, but this issue will become more prominent once we fix [JDK-8285447](https://bugs.openjdk.org/browse/JDK-8285447). The gory details follow below:
>> 
>> The protocol between the Java API and the JVM for `StackWalker.getCallerClass()/walk()` is as follows:
>> - On the Java side, `StackWalker` calls into `StackStreamFactory` for the real work.
>> - For `StackWalker.getCallerClass()` `StackStreamFactory` basically creates a `Class[]` which will be passed down and filled in the JVM. For `StackWalker.walk()` it will normally be a `StackFrameInfo[]` (or a `LiveStackFrameInfo[]` if the internal `ExtendedOption.LOCALS_AND_OPERANDS` option was used).
>> - The default size of this arrays is currently `StackStreamFactory.SMALL_BATCH` which is 8 (but see [JDK-8285447](https://bugs.openjdk.org/browse/JDK-8285447)).
>> - `StackStreamFactory` than calls `AbstractStackWalker.callStackWalk()` which is a natively implemented in the VM by `JVM_CallStackWalk()`.
>> -  `JVM_CallStackWalk()` calls `StackWalk::walk()` which calls `StackWalk::fetchFirstBatch()` which calls `StackWalk::fill_in_frames()` which walks the stack and fills in the available class/stackframe slots in the passed in array until the array is full or there are no more stack frames,
>> - Once  `StackWalk::fill_in_frames()` returns, `StackWalk::fetchFirstBatch()` calls back to Java by invoking `AbstractStackWalker::doStackWalk()` to consume the result.
>> - `AbstractStackWalker::doStackWalk()` calls `consumeFrames()` (which is overridden depending on whether we initially called `getCallerClass()` or `walk()`) which consumes the frames until it either finishes (e.g. finds the caller class) or until there are no more frames.
>> - In the latter case `consumeFrames()` will call into the the VM again by calling `AbstractStackWalker.fetchStackFrames()` to fetch additional frames from the stack.
>> - `AbstractStackWalker.fetchStackFrames()` is implemented by `JVM_MoreStackWalk()` which calls `StackWalk::fetchNextBatch()` which calls `StackWalk::fill_in_frames()` (the same method that already fetched the initial batch of frames).
>> 
>> Following is a stacktrace of what I've explained so far:
>> 
>> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
>> V  [libjvm.so+0x1...
>
> Volker Simonis has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Rename new parameter according to the HS coding conventions

Thanks for catching this issue.    I agree that `Method::invoke` should be skipped the caller-sensitive test in this case but the fix isn't quite right.    The caller-sensitive test should apply in any batch.    For example, `CSM` calls `getCallerClass` reflectively, I think the stack would look like this:


java.lang.StackWalker::getCallerClass
java.lang.invoke.DirectMethodHandle$Holder::invokeStatic
java.lang.invoke.LambdaForm$MH/0x0000000800002c00::invoke
:
:
jdk.internal.reflect.DirectMethodHandleAccessor::invokeImpl
jdk.internal.reflect.DirectMethodHandleAccessor::invoke
java.lang.reflect.Method::invoke
CSM  <--------- caller-sensitive method and UOE should be thrown



In this case, UOE should be thrown.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/14773#issuecomment-1622325630