RFR: 8189871: Refactor GC barriers to use declarative semantics

Fri Nov 10 16:29:04 UTC 2017

Hi Roman,

On 2017-11-10 16:01, Roman Kennke wrote:
> Hi Erik,
>
> This looks very good to me. It is likely that we'll need to extend it 
> a little bit for Shenandoah, but I haven't got around to try that out 
> yet, and will propose it when this patch percolated down to the 
> Shenandoah project.

Yes. The framework should be quite flexible, and of course I will work 
with you on anything that needs to be updated.

> Questions (I know I've asked some of it before in private discussions):
> - A BarrierSet needs to declare an AccessBarrier inner class. How does 
> this get 'registered' with the Access dispatcher?

Good question. Each new GC is tied together at one single point. The 
Access API picks up GCs from the gc/shared/barrierSetConfig.hpp and 
.inline.hpp files.

So to register a new GC, such as Shenandoah, you have to:

1) Make sure you have a BarrierSet enum value which is added to the list 
of FOR_EACH_BARRIER_SET_DO as well as FOR_EACH_CONCRETE_BARRIER_SET_DO 
in barrierSetConfig.hpp.

The first of said lists contains all barrier sets that are known to 
exist at build time, and the second of said lists crucially contains 
concrete barrier sets that have an AccessBarrier to resolve.

2) You also need to make sure in the barrierSetConfig.inline.hpp file 
that you #include your shenandoah BarrierSet inline.hpp file.

3) You have to provide a specialization for the BarrierSet::GetName and 
BarrierSet::GetType metafunctions that provide an enum value for a 
barrier set type, and vice versa.

4) Since you probably want primitive accesses in the heap to also 
resolve into the barrier set in a build that includes Shenandoah, you 
should #define SUPPORT_BARRIER_ON_PRIMITIVES in the barrierSetConfig.hpp 
file when building with Shenandoah. This will flick on the 
INTERNAL_BT_BARRIER_ON_PRIMITIVES decorator to each access so that the 
Access framework understands that even primitive accesses must be 
resolved at run-time in the barrier set. So this is a build-time switch 
for turning on run-time resolution of primitive accesses in the heap.

And now you should be set: your new ShenandoahBarrierSet::AccessBarrier 
will be called for each access, including primitives.

It works the following way:

1) The barrier resolver loads the current barrier set, and checks the 
"name" of it (the enum value).
2) Each "name" for concrete barriers that you listed in 
barrierSetConfig.hpp is asked for...
3) ...the BarrierSet::GetType of that enum "name", and...
4) The AccessBarrier of that resulting BarrierSet (your 
ShenandoahBarrierSet) will be called.

Hope that makes sense.

> - I see you use namespace. I haven't seen them anywhere else in 
> Hotspot, so this looks quite unusual :-) Not that I am against it (I 
> would probably advocate for using more of it), but have you considered 
> alternatives that look more common Hotspot-style (e.g. declaring an 
> all-static AccessInternal class)?

We do not have any platform that has problems with namespaces. So I 
would prefer this basic namespace usage to a bunch of AllStatic classes. 
I hope nobody minds that.

> - The dispatching machinery looks a bit over the top, and from the 
> outskirts like a manual re-invention of virtual method dispatch. 
> Couldn't we do the same stuff with the usual public interface / 
> concrete implementation idioms? I am worried that adding just one 
> method to the interface turns into a nightmare of wiring up stuff and 
> adding tons of boilerplate to get it going. Not to mention the 
> learning curve involved trying to make sense of what goes where.

It's not quite like a normal virtual call though. It's a virtual 
dispatch that carries template parameters through the runtime dispatch. 
The template parameters are required for 1) remembering the decorators 
(semantics of the access), and 2) the type information of the operands 
for the access.

In my experience, when you need to drag template parameters through some 
form of virtual dispatch, it is easiest to use function pointers and not 
virtual calls.
I understand it might look a bit complicated, but I hope that is okay.

> Other than that, I very much like what I see.

I'm glad to hear it!

Thanks,
/Erik

> Roman
>
>> Hi,
>>
>> In an effort to remove explicit calls to GC barriers (and other 
>> orthogonal forms of barriers, like encoding/decoding oops for 
>> compressed oops and fencing for memory ordering), I have built an API 
>> that I call "Access". Its purpose is to perform accesses with 
>> declarative semantics, to handle multiple orthogonal concerns that 
>> affect how an access is performed, including memory ordering, 
>> compressed oops, GC barriers for marking, reference strength, etc, 
>> and as a result making GCs more modular, and as a result allow new 
>> concurrently compacting GC schemes utilizing load barriers to live in 
>> harmony in hotspot without everyone going crazy manually inserting 
>> barriers if UseBlahGC is enabled.
>>
>> CR:
>> https://bugs.openjdk.java.net/browse/JDK-8189871
>>
>> Webrev:
>> http://cr.openjdk.java.net/~eosterlund/8189871/webrev.00/
>>
>> So there are three views of this I suppose:
>>
>> 1) The frontend: how this is actually used in shared code
>> 2) The backends: how anyone writing a GC sticks their required 
>> barriers in there
>> 3) The internals: how accesses find their way from the frontend to 
>> the corresponding backend
>>
>> == Frontend ==
>>
>> Let's start with the frontend. I hope I made this fairly simple! You 
>> can find it in runtime/access.hpp
>> Each access annotates its declarative semantics with a set of 
>> "decorators", which is the name of the attributes/properties 
>> affecting how an access is performed.
>> There is an Access<decorator> API that makes the declarative 
>> semantics possible.
>>
>> For example, if I want to perform a load acquire of an oop in the 
>> heap that has "weak" strength, I would do something like:
>> oop result = Access<MO_ACQUIRE | IN_HEAP | 
>> ON_WEAK_OOP_REF>::oop_load_at(obj, offset);
>>
>> The Access API would then send the access through some GC backend, 
>> that overrides the whole access and tells it to perform a "raw" load 
>> acquire, and then possibly keep it alive if necessary (G1 SATB 
>> enqueue barriers).
>>
>> To make life easier, there are some helpers for the most common 
>> access patterns that merely add some default decorator for the 
>> involved type of access. For example, there is a RawAccess for 
>> performing AS_RAW accesses (that bypasses runtime checks and GC 
>> barriers), HeapAccess sets the IN_HEAP decorator and RootAccess sets 
>> the IN_ROOT decorator for accessing root oops. So for the previous 
>> call, I could simply do:
>>
>> oop result = HeapAccess<MO_ACQUIRE | 
>> ON_WEAK_OOP_REF>::oop_load_at(obj, offset);
>>
>> The access.hpp file introduces each decorator (belonging to some 
>> category) with an explanation what it is for. It also introduces all 
>> operations you can make with access (loads, stores, cmpxchg, xchg, 
>> arraycopy and clone).
>>
>> This changeset mostly introduces the Access API but is not complete 
>> in annotating the code more than where it gets very awkward if I don't.
>>
>> == Backend ==
>>
>> For a GC maintainer, the BarrierSet::AccessBarrier is the top level 
>> backend that provides basic accesses that may be overridden. By 
>> default, it just performs raw accesses without any GC barriers, that 
>> handle things like compressed oops and memory ordering only. The 
>> ModRef barrier set introduces the notion of pre/post write barriers, 
>> that can be overridden for each GC. The CardTableModRef barrier set 
>> overrides the post write barrier to mark cards, and G1 overrides it 
>> to mark cards slightly differently and do some SATB enqueueing. G1 
>> also overrides loads to see if we need to perform SATB enqueue on 
>> weak references.
>>
>> The raw accesses go to the RawAccessBarrier (living in 
>> accessBackend.hpp) that performs the actual accesses. It connects to 
>> Atomic and OrderAccess for accesses that require that.
>>
>> == Internals ==
>>
>> Internally, the accesses go through a number of stages in 
>> access.inline.hpp as documented at the top.
>>
>> 1) set default decorators and get rid of CV qualifiers etc. Sanity 
>> checking also happens here: we check that the decorators make sense 
>> for the access being performed, and that the passed in types are not 
>> bogus.
>> 2) reduce types so if we have a different type of the address and 
>> value, then either it is not allowed or it implies we use compressed 
>> oops and remember that we know something about whether compressed 
>> oops are used or not, before erasing address type
>> 3) pre-runtime dispatch: figure out if all runtime checks can be 
>> bypassed into a raw access
>> 4) runtime dispatch: send the access through a function pointer that 
>> upon the first invocation resolves the intended GC AccessBarrier 
>> accessor on the BarrierSet that handles this access, as well as 
>> figures out whether we are using compressed oops or not while we are 
>> at it, and then calls it through the post-runtime dispatch
>> 5) post-runtime dispatch: fix some erased types that were not known 
>> at compile time such as whether the address is a narrowOop* or oop* 
>> depending on whether compressed oops was selected at runtime or not, 
>> and call the resolved BarrierSet::AccessBarrier accessor 
>> (load/store/etc) with all the call-site build-time and run-time 
>> resolved decorators and type information that describes the access.
>>
>> Testing: mach5 tier1-5
>>
>> Thanks,
>> /Erik
>
>