Shenandoah hangs on specjvm2008 due to reentrant handshake

[Zhengyu Gu]

Hi Liang,


On 5/12/21 6:53 AM, Liang Mao wrote:
> Hi Shenandoah team,
> 
> We found this issue while running Shenandoah with jdk/master:
> https://bugs.openjdk.java.net/browse/JDK-8266963 in which we had a simple fix
> so far.

Thanks for reporting. Could you please post your fix?

> 
> The hanging comes from https://bugs.openjdk.java.net/browse/JDK-8262443 which
> introduced the possibility of reentrant handshake and looks like only shenandoah
> would hit the scenario. I have a question specific to shenandoah that why do we
>   need the phase of "entry_update_thread_roots"?
> I think we already concurrently relocate the oops in thread roots by
> "entry_thread_roots".

entry_thread_roots fixes on-stack frames, but can not prevent thread 
from loading from-space oops during continuing execution. Ideally, 
Shenandoah LRB should prevent this from happening, if LRB is "atomic" ( 
no safepoints between load and LRB), unfortunately, current LRB is not.

> And "entry_update_thread_roots" doesn't use a general concurrent stack processing
> via stack watermark but just a handshake.

At point we execute entry_update_thread_roots, cset has been completely 
evacuated, therefore, no more from-space oops in heap. There are very 
rare cases that threads can still hold stalled oops, can be flushed out 
very quick, especially, if we can handshake individual Java thread and 
not cause global safepoint.

-Zhengyu

> 
> Thanks,
> Liang
>