RFR: 8324969: C2: prevent elimination of unbalanced coarsened locking regions [v6]

[Vladimir Kozlov]

On Fri, 16 Feb 2024 17:34:10 GMT, Vladimir Kozlov <kvn at openjdk.org> wrote:

>> src/hotspot/share/opto/compile.cpp line 4991:
>> 
>>> 4989:           }
>>> 4990:         }
>>> 4991:         assert(box->is_unbalanced() == box->is_marked_unbalanced(),"inconsistency");
>> 
>> Could you even move this a scope out, i.e. a line down, so this check is even run with `alock->is_coarsened() == false`?
>
> Right. It will verify that we did not miss `is_unbalanced()` check when do EA or nested elimination to avoid them.
> 
> I will also move the assert at line 4984 under `(box != this_box)` to avoid duplication of check when `box` is the same.

Moving assert down does not work because Nested lock elimination and EA can overwrite Coarsened status for BoxLock before we run `mark_unbalanced_boxes()`. Such BoxLock will not be marked as Unbalanced and assert will fail.
I would like to keep current assert in place but add an other assert when we change status of BoxLock. I am working on it.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/17697#discussion_r1496236285