Shenandoah hangs on specjvm2008 due to reentrant handshake

[Liang Mao]

Hi Zhengyu,

> entry_thread_roots fixes on-stack frames, but can not prevent thread 
> from loading from-space oops during continuing execution. Ideally, 
> Shenandoah LRB should prevent this from happening, if LRB is "atomic" ( 
> no safepoints between load and LRB), unfortunately, current LRB is not.

Is there a chance that threads will load stale oop in from-space  and
hold it after fix-ref is down and then store the stale oop into heap before
entry_update_thread_roots?

Is there a plan to re-use the barrier data to implement LRB to completely
avoid "non-atomic" of loads and LRB like ZGC?

Thanks,
Liang

------------------------------------------------------------------
From:Zhengyu Gu <zgu at redhat.com>
Send Time:2021 May 12 (Wed.) 20:25
To:"MAO, Liang" <maoliang.ml at alibaba-inc.com>; shenandoah-dev <shenandoah-dev at openjdk.java.net>
Subject:Re: Shenandoah hangs on specjvm2008 due to reentrant handshake

Hi Liang,


On 5/12/21 6:53 AM, Liang Mao wrote:
> Hi Shenandoah team,
> 
> We found this issue while running Shenandoah with jdk/master:
> https://bugs.openjdk.java.net/browse/JDK-8266963 in which we had a simple fix
> so far.

Thanks for reporting. Could you please post your fix?

> 
> The hanging comes from https://bugs.openjdk.java.net/browse/JDK-8262443 which
> introduced the possibility of reentrant handshake and looks like only shenandoah
> would hit the scenario. I have a question specific to shenandoah that why do we
>   need the phase of "entry_update_thread_roots"?
> I think we already concurrently relocate the oops in thread roots by
> "entry_thread_roots".

entry_thread_roots fixes on-stack frames, but can not prevent thread 
from loading from-space oops during continuing execution. Ideally, 
Shenandoah LRB should prevent this from happening, if LRB is "atomic" ( 
no safepoints between load and LRB), unfortunately, current LRB is not.

> And "entry_update_thread_roots" doesn't use a general concurrent stack processing
> via stack watermark but just a handshake.

At point we execute entry_update_thread_roots, cset has been completely 
evacuated, therefore, no more from-space oops in heap. There are very 
rare cases that threads can still hold stalled oops, can be flushed out 
very quick, especially, if we can handshake individual Java thread and 
not cause global safepoint.

-Zhengyu

> 
> Thanks,
> Liang
>