RFR: 8266963: Reentrance condition for safepoint/handshake

[Robbin Ehn]

On Fri, 14 May 2021 08:01:12 GMT, Yude Lin <github.com+16811675+linade at openjdk.org> wrote:

> Shenandoah hangs when running specjvm2008 derby. The reason is a Java Thread reenters safepoint/handshake and blocks on itself. Please checkout the bugid for more details. After discussion with @zhengyu123, we think this might not be Shenandoah-specific. I propose to add a check before processing the safepoint/handshake.
> 
> An alternative approach (also insight from @zhengyu123) is to move the check a little earlier to the specific place where the Java Thread do ThreadBlockInVM. To feel reassured that no more reentrance exists in other places, I still leave the check in safepoint/handshake as debug code. See https://github.com/openjdk/jdk/compare/master...linade:reentrancecond
> 
> I'd appreciate more of your thoughts on these as I understand it could be a rather critical part of the code.

We can just remove this:

    if (i != 0 && thread->is_Java_thread()) {
      JavaThread* jt = thread->as_Java_thread();
      if (jt->thread_state() == _thread_in_vm) {
        // Since this JavaThread has looped at least once and is _thread_in_vm,
        // we honor any pending blocking request.
        ThreadBlockInVM tbivm(jt);
      }
    }



Under some wired circumstances it can prolong TTS.
But it was me that found it. Reverting it would get any complains from anyone else.

(we have seen this in experimental code internally, and just remove code which goes to blocked was suggested then also)

-------------

PR: https://git.openjdk.java.net/jdk/pull/4028