RFR: 8079315: UseCondCardMark broken in conjunction with CMS precleaning

[Erik Österlund]

Hi Andrew,

I have heard statements like this that such mechanism would not work on RMO, but never got an explanation why it would work only on TSO. Could you please elaborate?
I studied some kernel sources for a bunch of architectures and kernels, and it seems as far as I can see all good for RMO too.
Even PPC does the following on each relevant CPU when flushing TLBs from mprotect: sync; purge_tlbs; isync; Therefore it’s not just serializing stores but flushing loads also, making it pretty similar on both TSO and RMO.

I don’t see any problematic reordering that would make this technique unreliable on RMO machines. Can you?

1: x.a = something
2: StoreStore
3: if (card[@x.a] != dirty) {
4:  card[@x.a] = dirty
5: }

In the following example (the problem at hand), which RMO reordering could harm us? The one relevant reordering that can happen that I can see (and is what this whole issue is all about) is that the store on line 1 is delayed and reorders with the conditional mark check on line 3. The global fence technique would guarantee that in the event of such a reordering, both the store write (line 1) and conditional mark check (line 3) will both be observed as-if they did not reorder because they will both be serialized and globally observed before any consistency is violated. 

To be more exact, the violation would be that the cleaning thread would after cleaning, read a stale reference (from line 1) that was deferred in the store buffer while the card is being precleaned while the mutator assumes there is no need to dirty the card again. If every processor has to issue a full fence which kernel sources imply (e.g. sync on PPC), then this consistency violation is equally impossible even on RMO as far as I can see.

Thanks,
/Erik

> On 11 May 2015, at 10:52, Andrew Haley <aph at redhat.com> wrote:
> 
> On 05/10/2015 09:58 PM, David Holmes wrote:
> 
>> For background on memory serialization page see Dave Dice's blog entries:
>> 
>> https://blogs.oracle.com/dave/entry/qpi_quiescence
>> 
>> https://blogs.oracle.com/dave/resource/Asymmetric-Dekker-Synchronization-140215.txt
> 
> Note that this approach isn't safe except on TSO.  We'll still
> need some kind of barriers for everyone else.
> 
> Andrew.
>