[aarch64-port-dev ] AARCH64 stage: next steps

[Bertrand Delsart]

On 23/01/2015 11:09, Andrew Haley wrote:
> On 23/01/15 09:55, David Holmes wrote:
>> On 23/01/2015 7:29 PM, Andrew Haley wrote:
>>> On 23/01/15 02:10, David Holmes wrote:
>>>> On 22/01/2015 9:17 PM, Andrew Haley wrote:
>>>>> On 01/22/2015 10:12 AM, David Holmes wrote:
>>>>>> Yes this is a badly documented flag. While originally (when we only had
>>>>>> TSO systems) it allowed you to switch between the memory serialization
>>>>>> pseudo-membar trick and full membars/fences,  you need UseMembar for any
>>>>>> platform for which the memory serialization trick is not guaranteed to
>>>>>> work. As we have discussed before we consider that to be any non-TSO
>>>>>> platform, but PPC64 decided otherwise (did Aarch64 do the same?). :)
>>>>>
>>>>> UseMembar is enabled by default on AArch64.
>>>>
>>>> Then you don't need to use the thread_state()/set_thread_state()
>>>> variants that PPC64 uses.
>>>
>>> Sorry, I still don't get it.  What's the connection?  The thread state
>>> change needs to be communicated from the mutator thread to the
>>> collector thread.  Without an acquire in thread_state() the collector
>>> won't see the state change.  Also, the thread state is set in three
>>> places during a native method but there is only one write to the
>>> serialization page.
>>
>> What are the mutator threads and collector threads you are referring to
>> here? Sorry if I missed the context. The primary use of thread_state is
>> for managing the safepoint mechanism, which is done via the
>> interfaceSupport routines, which in turn uses (potentially) the
>> serialization page to ensure communication, or else membars. The
>> accesses to thread_state within those routines should not need any
>> additional barriers.
>
> But what is the point of changing thread_state is no other thread
> needs to see it?
>
>> As I said during the PPC64 review if there is some other code that is
>> utilizing thread_state separately then that code may need additional
>> barriers. My objection was to putting them inside the accessors because
>> they were then used in contexts where they were not needed.
>
> Okay, I'll ask the question explicitly: if only a single thread_state
> change needs to be visible, why change it three times?

The code was optimized by people deciding exactly how the mutator 
threads would update their state and how the VM threads would read the 
state of the mutator threads while performing a safepoint. It indeed 
involves complex mechanisms... which were highly tuned (and tested) over 
the years and which have now been stable for a long time.

We need some barriers at the right places and these barriers are not 
just a lock_acquire when reading the state and a release_store when 
modifying it. The load_acquire/release_store end up being redundant with 
other operations performed when UseMembar is true.

For instance, note that SafepointSynchronize::do_call_back() does not 
involve a load_acquire. Correctness of the safepointing mechanism relies 
on a fence (or the serialization page mechanism) between the change of 
the state and the do_call_back read. Those who designed the safepointing 
code and improved it for non-TSO systems tried to ensure that this was 
the only required barrier for the mutator thread.

In fact, this is why we have several state changes. The fence AFTER the 
first one is not sufficient to ensure that any change happening BEFORE 
state change is visible to the GC. Now, the fence above acts as a 
release before the second state update. The GC will wait for that second 
update, hence being sure that everything that happened before the 
transition is visible.

I hope that this tip is sufficient to allow you to understand the 
safepointing mechanism by yourself. Explaining it all by e-mail would be 
very difficult :-)

The good news is that you use UseMembar for AARCH64. I think that those 
who reviewed PPC64 changes were still dubious about the fact that the 
serialization page mechanism works for non-TSO systems. However, they 
did not object at review time as long as the proposed changes did not 
impact other platforms. These changes would have to be adapted if 
someone else wanted to provide a PPC64 port relying on UseMembar.

We are now supporting two different orderings policies for non-TSO 
systems in the transition code. This has impacts on other pieces of 
code, sometimes requiring extra barriers only for one of these policies. 
We are trying to avoid this overhead for the hot paths... and 
set_java_state is one of these hot paths.

Regards,

Bertrand.

>
> Andrew.
>


-- 
Bertrand Delsart,                     Grenoble Engineering Center
Oracle,         180 av. de l'Europe,          ZIRST de Montbonnot
38330 Montbonnot Saint Martin,                             FRANCE
bertrand.delsart at oracle.com             Phone : +33 4 76 18 81 23

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
NOTICE: This email message is for the sole use of the intended
recipient(s) and may contain confidential and privileged
information. Any unauthorized review, use, disclosure or
distribution is prohibited. If you are not the intended recipient,
please contact the sender by reply email and destroy all copies of
the original message.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~