Forwarding pointer elimination questions

[Roman Kennke]

Hi Simone,

> [new subject to avoid hijacking your RFR one]
> 
> On Wed, May 8, 2019 at 11:14 AM Aleksey Shipilev <shade at redhat.com> wrote:
>> https://mail.openjdk.java.net/pipermail/shenandoah-dev/2019-April/009203.html
>>
>> LRB = LoadReferenceBarrier. LRB intercepts heap accesses at load-reference time, evacuates the
>> object then.
> 
> This is quite similar to the ZGC and C4 barrier, right?

Not quite. ZGC and C4 also use the load-barriers to support marking. 
Shenandoah still has (and will have) SATB barriers for that. Also, the 
technique used to enforce the to-space-invariant is totally different.

> Is the LRB used only during evacuation, or also in the other phases
> (marking, update_reference)?

LRB is active whenever there are forwarded objects. That is evacuation, 
update-references, and marking if marking also does piggy-backed 
update-references.

A good way to think about this is:
Any object that is in-use has the lifecycle:
1. It gets loaded or created somewhere
2. It gets passed around
3. It is used for various things (reading from, writing to, comparing, etc)
eventually it goes out of scope

We used to employ the barriers, and ensure a reasonable invariant at 
use-sites (e.g. #3). Now we're employ the barriers and enforce *strong* 
invariant right when the object enters the system (#1). The advantages are:
1. Strong invariant means we don't need to worry about that later, e.g. 
when comparing or storing objects
2. Optimization is easier: placement at load-site is naturally less hot 
than placement at use-site. Our previous optimizations worked very hard 
to move barriers away from use-sites. Now we're basically starting out 
with almost-optimal placement and don't need to do very much.
3. Since all barriers are now basically what used to be write-barriers, 
and because those are conditional on GC phase and object-status (e.g. 
part of cset or not), the actual slow-path is not all that hot and it 
doesn't hurt to e.g. complicate the forward pointer resolution a tiny 
little bit by introducing decoding the mark word for it.

> Are you storing metadata on the side when the mutator triggers the
> LRB, so that the CAS is now performed on the side data structure
> rather than the forwarding pointer?

No. The CAS is performed on the mark word of the from-space copy. The 
protocol is not significantly different than before.

> What's the LRB looking like in ASM?

It's pretty much like the old WB.

> Feels like you just moved from a Brooks pointer algorithm to a C4-like
> algorithm (as described in the C4 paper - perhaps not exactly the same
> as C4 or ZGC implementation wise).

The actual algorithm is still the same: Evacuation performs an 
speculative copy of the object, then competes to CAS the forwarding 
pointer. The only difference is that before we CASed it into the extra 
slot, now it's CASed to the mark word slot.

The reason why LRB enables to do this is that in the old world, we'd 
have to do barriers on use-sites. In particular, we needed read-barriers 
when reading from fields. Those read-barriers tended to be very hot. 
Complicating those read-barriers by decoding a mark word would have been 
prohibitive. (I had a prototype for that, and it was no good.)

> If so, I have to update my slides :)

A little bit, yeah. OTOH, LRB and elimination for forward pointer word 
are quite new (jdk13) and only slowly coming down the pipeline (jdk12, 
jdk11 and jdk8).

Does the clarify the situation?

Roman