RFR: Process remaining SATB buffers in final mark/traverse loop instead of separate phase

[Roman Kennke]

Am 19.06.2018 um 19:45 schrieb Aleksey Shipilev:
> On 06/19/2018 07:22 PM, Roman Kennke wrote:
>> We don't need an extra phase to process remaining complete buffer in
>> final-mark/-traverse, and we don't need all the loop versions twice with
>> and without SATB. We can just as well do that in one single loop, pretty
>> much the same as conc-mark/-traversal. The important part is to process
>> *in*complete thread-local buffers. We already piggy-back this on
>> final-thread-scanning in traversal, and we keep a small extra phase for
>> that in conc-mark. At the very least, this should keep the overall
>> generated code smaller.
>>
>> Aleksey, can you give this some smoke performance runs to check for
>> possible regressions or maybe improvements?
>>
>> http://cr.openjdk.java.net/~rkennke/satb-final-phase/webrev.00/
> 
> Looks good, except... The thing about code like below, is that it is deliberately racy: we are
> checking completed_buffers_num without taking the SATB mutex:
> 
> 1002     while (satb_mq_set.completed_buffers_num() > 0) {
> 1003       satb_mq_set.apply_closure_to_completed_buffer(&drain_satb);
> 1004     }
> 
> ...which means, we can miss the complete buffer while there is actually one. It does not matter for
> the concurrent cycle, because we would either see it on next marking stride, or final-* would drain
> everything without checking for this flag.
> 
> But this changes once we remove this line from SCM::drain_satb_buffers():
> 
> 507  while (satb_mq_set.apply_closure_to_completed_buffer(&cl));
> 
> Now, we at the mercy of that racy check to drain the completed buffers. I would not vouch the whole
> thing works correctly then. It seems that leaving that line alone provides us with guarantees for
> final-mark. Not sure if the same applies to Traversal.
> 
> So, I think we can drop DRAIN_SATB template parameter and simplify marking loops then, but not
> really remove the unconditional SATB draining that guarantees finishing SATBs.

Note that during final-* no mutators are racing with us. Which means
that num-completed-buffers only ever decrements. Can we see 0 there when
there are remaining buffers left?

Roman