RFR: 8162929: Enqueuing dirty cards into a single DCQS during GC does not scale

[Thomas Schatzl]

Hi,

On Fri, 2019-07-12 at 14:36 +0200, Thomas Schatzl wrote:
> Hi,
> 
> On Fri, 2019-07-05 at 00:50 -0400, Kim Barrett wrote:
> > Please review this refactoring of G1DirtyCardQueue[Set], splitting
> > into two sets of classes that each better support the purpose they
> > are used for.  The existing G1DirtyCardQueue[Set] is (and remains)
> > the mechanism for communicating between the logging post-write
> > barrier and the remset, via refinement (both concurrent and
> > STW).  The new G1RedirtyCardsQueue[Set] are used during collections
> > to collect and en masse dump old remset information back into that
> > cycle for later reconsideration.
> > 
[...]
> > CR:
> > https://bugs.openjdk.java.net/browse/JDK-8162929
> > 
> > Webrev:
> > http://cr.openjdk.java.net/~kbarrett/8162929/open.00/
> > 
> 
> some comments:
> 
> 
> [...]
>
> - it's a bit unfortunate that on the way from the redirty qset to the
> global qsets (in G1DirtyCardQueueSet::merge_bufferlists) we can not
> easily keep a "tail" pointer and count, and then when merging the
> redirty qset with the dirty qset have to iterate over the BufferNodes
> to determine it.
> 
> This might have some minor (seemingly unnecessary) impact on huge
> loads like we discussed internally when doing perf testing. Probably
> not worth the effort dealing with here, given that you may simply
> increase the buffer size there.
> I will do some cursory tests, with probably no detrimental outcome
> though.

Merging the bufferlists (adding a GCtraceTime scoped object before the
call) adds up to ~2.5ms, that results in up to 0.5% of total pause
time, on BigRAMTester 50G with 1M regions (a somewhat contrived example
in many ways).
It's not much better with 32M regions either unfortunately, as GC
itself is faster there (around ~0.3% of total pause time, little bit
less contrived).

Is there a way to improve that?

(The previous merge code took in the single digit us range)

Thanks,
  Thomas