Workers count boosting after Mark Start

[Pierre Mevel]

Good morning,

Apologies to bother you once again. As i explained in an earlier email (
https://mail.openjdk.java.net/pipermail/zgc-dev/2019-September/000736.html),
I'm seeing a lot of Allocation Stalls using ZGC, the issue being that my
application's allocation rate is quite high.

I have GC cycles running back to back, whatever the value of ConcGcThreads.
(The amount of application threads being superior to the amount of vCPUs, I
even tried things like 40 ConcGcThreads on a 20 vCPUs machine, but at some
point I don't want context switching to be too big of a factor).

However I never see, in debug mode, the workers count boosting log lines.
Looking at the code (correct me if I'm wrong), I'm seeing that only at the
beginning of a cycle can the workers count be boosted to
Max(ParallelGcThreads, ConcGcThreads), as it happens in the do_operation
method of the VM_ZMarkStart class.

For the allocation threads to be stalled at Mark Start, it should mean that
Heap Memory is full at the start of the cycle. Since the cycles start
preemptively with timeUntilGC, etc..., it only happens if the cycles run
back to back, and heap memory is full at the start of the cycle, meaning
that allocation rate was higher than the "freeing rate" during concurrent
relocation.

In my case, allocation stalls appear during concurrent mark.

Would it be possible, in theory, for the `should_boost_worker_threads`
method from zDriver to be called within `concurrent_mark_continue`, change
the amount of workers and continue the ZMarkTask with more workers?

Best regards and thanks in advance for your answer,

Pierre Mével
pierre.mevel at activeviam.com
ActiveViam
46 rue de l'arbre sec, 75001 Paris