Shenandoah hangs on specjvm2008 due to reentrant handshake

[Zhengyu Gu]

> 
> Thanks for reporting. Could you please post your fix?
> 
Saw your fix in CR.

Thanks,

-Zhengyu

>>
>> The hanging comes from 
>> https://bugs.openjdk.java.net/browse/JDK-8262443 which
>> introduced the possibility of reentrant handshake and looks like only 
>> shenandoah
>> would hit the scenario. I have a question specific to shenandoah that 
>> why do we
>>   need the phase of "entry_update_thread_roots"?
>> I think we already concurrently relocate the oops in thread roots by
>> "entry_thread_roots".
> 
> entry_thread_roots fixes on-stack frames, but can not prevent thread 
> from loading from-space oops during continuing execution. Ideally, 
> Shenandoah LRB should prevent this from happening, if LRB is "atomic" ( 
> no safepoints between load and LRB), unfortunately, current LRB is not.
> 
>> And "entry_update_thread_roots" doesn't use a general concurrent stack 
>> processing
>> via stack watermark but just a handshake.
> 
> At point we execute entry_update_thread_roots, cset has been completely 
> evacuated, therefore, no more from-space oops in heap. There are very 
> rare cases that threads can still hold stalled oops, can be flushed out 
> very quick, especially, if we can handshake individual Java thread and 
> not cause global safepoint.
> 
> -Zhengyu
> 
>>
>> Thanks,
>> Liang
>>