[aarch64-port-dev ] Fwd: Re: AARCH64 stage: next steps

[Bertrand Delsart]

Resending to aarch64-port-dev now that I registered (since I do not know 
how long it would take for the moderator to approve the previous message).

If you want to answer before the original one is unblocked, please add 
the other recipients to your answer.

Bertrand.

-------- Forwarded Message --------
Subject: Re: [aarch64-port-dev ] AARCH64 stage: next steps
Date: Fri, 23 Jan 2015 13:17:51 +0100
From: Bertrand Delsart <bertrand.delsart at oracle.com>
To: Andrew Haley <aph at redhat.com>, David Holmes 
<david.holmes at oracle.com>, Lindenmaier, Goetz 
<goetz.lindenmaier at sap.com>, Dean Long <dean.long at oracle.com>, Vladimir 
Kozlov <vladimir.kozlov at oracle.com>, aarch64-port-dev at openjdk.java.net 
<aarch64-port-dev at openjdk.java.net>, 
hotspot-runtime-dev at openjdk.java.net <hotspot-runtime-dev at openjdk.java.net>

On 23/01/2015 11:09, Andrew Haley wrote:
> On 23/01/15 09:55, David Holmes wrote:
>> On 23/01/2015 7:29 PM, Andrew Haley wrote:
>>> On 23/01/15 02:10, David Holmes wrote:
>>>> On 22/01/2015 9:17 PM, Andrew Haley wrote:
>>>>> On 01/22/2015 10:12 AM, David Holmes wrote:
>>>>>> Yes this is a badly documented flag. While originally (when we only had
>>>>>> TSO systems) it allowed you to switch between the memory serialization
>>>>>> pseudo-membar trick and full membars/fences,  you need UseMembar for any
>>>>>> platform for which the memory serialization trick is not guaranteed to
>>>>>> work. As we have discussed before we consider that to be any non-TSO
>>>>>> platform, but PPC64 decided otherwise (did Aarch64 do the same?). :)
>>>>>
>>>>> UseMembar is enabled by default on AArch64.
>>>>
>>>> Then you don't need to use the thread_state()/set_thread_state()
>>>> variants that PPC64 uses.
>>>
>>> Sorry, I still don't get it.  What's the connection?  The thread state
>>> change needs to be communicated from the mutator thread to the
>>> collector thread.  Without an acquire in thread_state() the collector
>>> won't see the state change.  Also, the thread state is set in three
>>> places during a native method but there is only one write to the
>>> serialization page.
>>
>> What are the mutator threads and collector threads you are referring to
>> here? Sorry if I missed the context. The primary use of thread_state is
>> for managing the safepoint mechanism, which is done via the
>> interfaceSupport routines, which in turn uses (potentially) the
>> serialization page to ensure communication, or else membars. The
>> accesses to thread_state within those routines should not need any
>> additional barriers.
>
> But what is the point of changing thread_state is no other thread
> needs to see it?
>
>> As I said during the PPC64 review if there is some other code that is
>> utilizing thread_state separately then that code may need additional
>> barriers. My objection was to putting them inside the accessors because
>> they were then used in contexts where they were not needed.
>
> Okay, I'll ask the question explicitly: if only a single thread_state
> change needs to be visible, why change it three times?

The code was optimized by people deciding exactly how the mutator
threads would update their state and how the VM threads would read the
state of the mutator threads while performing a safepoint. It indeed
involves complex mechanisms... which were highly tuned (and tested) over
the years and which have now been stable for a long time.

We need some barriers at the right places and these barriers are not
just a lock_acquire when reading the state and a release_store when
modifying it. The load_acquire/release_store end up being redundant with
other operations performed when UseMembar is true.

For instance, note that SafepointSynchronize::do_call_back() does not
involve a load_acquire. Correctness of the safepointing mechanism relies
on a fence (or the serialization page mechanism) between the change of
the state and the do_call_back read. Those who designed the safepointing
code and improved it for non-TSO systems tried to ensure that this was
the only required barrier for the mutator thread.

In fact, this is why we have several state changes. The fence AFTER the
first one is not sufficient to ensure that any change happening BEFORE
state change is visible to the GC. Now, the fence above acts as a
release before the second state update. The GC will wait for that second
update, hence being sure that everything that happened before the
transition is visible.

I hope that this tip is sufficient to allow you to understand the
safepointing mechanism by yourself. Explaining it all by e-mail would be
very difficult :-)

The good news is that you use UseMembar for AARCH64. I think that those
who reviewed PPC64 changes were still dubious about the fact that the
serialization page mechanism works for non-TSO systems. However, they
did not object at review time as long as the proposed changes did not
impact other platforms. These changes would have to be adapted if
someone else wanted to provide a PPC64 port relying on UseMembar.

We are now supporting two different orderings policies for non-TSO
systems in the transition code. This has impacts on other pieces of
code, sometimes requiring extra barriers only for one of these policies.
We are trying to avoid this overhead for the hot paths... and
set_java_state is one of these hot paths.

Regards,

Bertrand.

>
> Andrew.
>


-- 
Bertrand Delsart,                     Grenoble Engineering Center
Oracle,         180 av. de l'Europe,          ZIRST de Montbonnot
38330 Montbonnot Saint Martin,                             FRANCE
bertrand.delsart at oracle.com             Phone : +33 4 76 18 81 23

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
NOTICE: This email message is for the sole use of the intended
recipient(s) and may contain confidential and privileged
information. Any unauthorized review, use, disclosure or
distribution is prohibited. If you are not the intended recipient,
please contact the sender by reply email and destroy all copies of
the original message.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~