RFR: Hole in CAS barrier when using traversal heuristics

[Zhengyu Gu]

>>>
>>> Hi Zhengyu,
>>>
>>> I am not sure what you mean by 'evacuates target and exchange object,
>>> but not the field' .. clearly the target object needs to be 
>>> evacuated, because we only write to to-space (write-barrier). Also, 
>>> the exchange object needs to be evacuated, to ensure we end up only 
>>> with to-space references in fields (storeval-barrier). What do you 
>>> mean by evacuation of 'the field' ? The target field is part of the 
>>> target object.
>>
>> There is an example:
>> http://hg.openjdk.java.net/shenandoah/jdk10/file/6183a72bd5c2/src/hotspot/share/prims/unsafe.cpp#l1020 
>>
>>
>> the addr points a field in object, that might not be evacuated and I 
>> think you do have to enqueue it, as the object may be gray.
> 
> addr is a field in p, which is in to-space by the WB a few lines above. 
> This should be good. x is the exchange value, also evacuated by the 
> storeval barrier. So all should be fine.

Yes, p and x are fine, but the field (e.g. an object) to be swapped out, 
may still in cset, and it is enqueued by 
oopDesc::atomic_compare_exchange_oop() 
(http://hg.openjdk.java.net/shenandoah/jdk10/file/3c12448ec444/src/hotspot/share/oops/oop.inline.hpp#l407), 
then hit assertion failure here:

http://hg.openjdk.java.net/shenandoah/jdk10/file/3c12448ec444/src/hotspot/share/gc/shenandoah/shenandoahBarrierSet.cpp#l464

hs_err: https://paste.fedoraproject.org/paste/isPRNXG6PqaPgx9a18IA4w


BTW, what's reason it has to be to-space object? can it be evacuated 
during processing SATB buffers, or by storeval_barrier()?


-Zhengyu



> 
> What error/assert/crash are you seeing? Is it something in 
> SBS::is_safe()? Then it may already be fixed by my subsequent changeset?
> 
> It may happen that traversal GC gets cancelled, and then we hit an 
> overly strict assert like that.
> 
> Roman