RFR: 8324174: assert(m->is_entered(current)) failed: invariant [v2]

[Vladimir Kozlov]

On Tue, 30 Jan 2024 21:20:13 GMT, Vladimir Kozlov <kvn at openjdk.org> wrote:

>> test/hotspot/jtreg/compiler/escapeAnalysis/TestNestedRelockAtDeopt.java line 30:
>> 
>>> 28:  * @requires vm.compMode != "Xint"
>>> 29:  * @run main/othervm -XX:-TieredCompilation -XX:-BackgroundCompilation -Xmx128M
>>> 30:  *                   -XX:CompileCommand=exclude,TestNestedRelockAtDeopt::main TestNestedRelockAtDeopt
>> 
>> Might it make sense to also have a run without flags, so that outside flags have more of an effect, and can trigger other shapes?
>
> Not with these changes. I want to keep this test to only verify that the issue is fixed.
> I will run additional testing with default flags but for this push I want to keep the tests as it is.

Testing with default flags caught an other issue. We compile `main()` method as OSR and when we deoptimize due to OOM in `test2` the `catch` is not executed and test exit with:

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "main"


I would like to file separate bug for it and push current changes without the test modification.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/17600#discussion_r1472074591