G1, Remembered Sets & Refinement

[Thomas Schatzl]

Hi,

On Mon, 2015-05-11 at 11:46 -0700, Yu Zhang wrote:
> Simon,
> 
> I will try to answer your questions.  
> Thanks,
> Jenny
> On 5/8/2015 8:11 AM, Simone Bordet wrote:
> 
> > Hi,
> > 
> > I would like to ask some clarification about remembered sets (RS),
> > buffers and refinement in G1.
> > 
> > My understanding is that G1 installs a write barrier to record old to
> > young pointers.

Actually all inter-region pointers, where old->young are special cases.

> > Let's assume that A is the object in old generation, and B is the
> > object in young generation.
> Yes.
> > 
> > I understand that when the barrier triggers, the card correspondent to
> > the place where the A resides is marked.
> > This card is then enqueued into a queue (the dirty card queue).
> yes
> > 
> > I understand that until the number of entries in the dirty card queue
> > does not enter the yellow zone, then nothing is done.
> > When the yellow zone is entered, a refinement thread is started to
> > poll items out of the dirty card queue and update the RS for the young
> > region.
> yes. The number of refinement threads activated is decided by
> G1ConcRefinementThresholdStep.

Actually refinement starts at the green threshold, until all refinement
threads are running at the yellow one. At the red threshold, mutator
threads start helping.

> > I understand that, when a young GC happens, the refinement threads are
> > stopped (if running), and "Update RS" phase takes care of processing
> > the dirty card queue.
> yes.
> > 
> > Provided my understanding is correct, what is the meaning of the word
> > "buffer" in this scenario ?
> > PrintGCDetails prints out a "Processed Buffers" subphase for "Update
> > RS", but what is a "buffer" ?
> as you mentioned, the buffer is a set of the dirty card
> queues(DirtyCardQueueSet).  The dirty cards are processed by
> concurrent refinement threads or at STW phase( update RS).
> > 
> > Another question: given that the write barrier knows exactly A's oop,
> > what is the reason for card marking, rather than just recording the
> > oop ?
> I hope others can chime in on this.  My guess it is related to memory
> footprint and performance.

- memory usage: a card covers a set of references which are often
changed together.
- performance: while a card is marked, that card is not re-enqueued
again. This kind of duplicate detection is simple using cards (or any
range of memory backed by an array), while hard otherwise.

> > Last question: would setting the yellow zone to zero always reduce the
> > "Update RS" to (almost) zero ? What problem could this setting
> > possibly generate ?
> Probably not.  It will push more work to concurrent refinement
> threads, but still could leave some work for STW phase.

- there will always be some work remaining to be done during the stw
pause
- this will effectively disable duplicate detection leading to much
higher cpu-usage as cards are more frequently re-processed. Often it is
actually advantageous to keep the cards longer in the queue.

Thanks,
  Thomas