Allocation pacing and graceful degradation in ShenandoahGC

Tue Nov 22 17:13:43 UTC 2022

Hi Alex,
I have spent some time understanding the Shenandoah Pacer and I will try to
answer your questions as best I can.

Does that mean that each Java thread goes throw runtime -> heap to
> allocate, and that's how pacer paces it? So we just pace any allocating
> thread and threads that allocate more will just hit this code more often.

Allocations from tlab do not go through pacer, but allocating a new tlab
does go through the pacer.  And yes a thread allocating more is more likely
to hit the pacer.

so I assume if there is no budget available it will pace a thread for up to
> 10ms, but it does not imply allocation failure.

Yes, it does not imply allocation failure. It is just a mechanism to ensure
concurrent gc is able to keep pace with the allocation rate.

 Heap class tries to allocate under lock and if unsuccessful considers this
> as allocation failure and handles it by calling ShenandoahControlThread.
> Does it mean that Pacer can’t cause GC to switch to degenerated mode or I
> am missing something?

Pacer itself does not cause GC to degenerate. It only delays the mutator
thread. As you mentioned earlier, after the expiry of wait time the mutator
thread would still attempt allocation which may succeed.
If the allocation rate is high, pacer may not be able to cope up, and in
that case the mutator thread may suffer allocation failure which would
result in running a degenerated GC cycle.

 If Pacer doesn't have budget to allocate memory it paces thread, but is
> there any global budget for pacing time or it is only per thread max
> (ShenandoahPacingMaxDelay)?

The wait time introduced by Pacer is per thread and bounded by
ShenandoahPacingMaxDelay. I don't think there is any global budget for
pacing time.

It would really nice if you can sched some light on these transitions:
> Concurrent Mode -> Pacing (single thread and total pacing time for all
> threads) -> most importantly logic of transitioning from pacing to
> degenerated GC

I will try to summarize the transition to degenerated GC.
Allocation failures are signaled by the mutator thread by setting a flag
_alloc_failure_gc [0] in ShenandoahControlThread::handle_alloc_failure()
and then it waits on _alloc_failure_waiters_lock [1] for notification from
the control thread
after it has handled the allocation failure.
The control thread executing run_service() checks if the flag
_alloc_failure_gc is set [2], if so it indicates a pending allocation
failure. It then tries to handle alloc failure by running either a
Degenerated GC or a Full GC cycle. That decision depends on
ShenandoahHeuristics::should_degenerate_cycle() which performs a simple
counter check for the number of consecutive degenerated GC cycles.

There are some details in the wiki [4] for pacing and degenerated gc in
case you have not looked at that.

I hope this helps you to move forward in your effort.

[0]
https://github.com/openjdk/jdk/blob/fba763f82528d2825831a26b4ae4e090c602208f/src/hotspot/share/gc/shenandoah/shenandoahControlThread.cpp#L531
[1]
https://github.com/openjdk/jdk/blob/fba763f82528d2825831a26b4ae4e090c602208f/src/hotspot/share/gc/shenandoah/shenandoahControlThread.cpp#L543
[2]
https://github.com/openjdk/jdk/blob/fba763f82528d2825831a26b4ae4e090c602208f/src/hotspot/share/gc/shenandoah/shenandoahControlThread.cpp#L100
[3]
https://github.com/openjdk/jdk/blob/fba763f82528d2825831a26b4ae4e090c602208f/src/hotspot/share/gc/shenandoah/shenandoahControlThread.cpp#L127
[4] https://wiki.openjdk.org/display/shenandoah/Main

Thanks,
Ashutosh Mehra

On Mon, Nov 14, 2022 at 4:07 PM Alex Dubrouski <adubrouski at linkedin.com>
wrote:

> Good afternoon everyone,
>
>
>
> I checked all video presentations and slides by Alex Shipilev and Roman
> Kennke about ShenandoahGC to find the answer for my question with no luck.
> I am trying to find more details about transitions between modes in
> ShenandoahGC
>
> I am looking for solution to assess concurrent collector health in real
> time using different metrics.
>
>
>
> Here is the schema of transitions, and allocation failure causes
> degenerated GC cycle, but it does not mention allocation pacing at all:
>
>
> https://github.com/openjdk/jdk/blob/fba763f82528d2825831a26b4ae4e090c602208f/src/hotspot/share/gc/shenandoah/shenandoahControlThread.cpp#L361
>
>
>
> I tried to dig further into this logic, but need your help to put all the
> pieces together
>
> I was not able to effectively trace entry point, but this might work,
> allocation on heap outside of TLAB:
>
>
> https://github.com/openjdk/jdk/blob/master/src/hotspot/share/gc/shared/memAllocator.cpp#L258
>
> in case of ShenandoahGC I assume we call
>
>
> https://github.com/openjdk/jdk/blame/739769c8fc4b496f08a92225a12d07414537b6c0/src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp#L901
>
> which then calls
>
>
> https://github.com/openjdk/jdk/blame/739769c8fc4b496f08a92225a12d07414537b6c0/src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp#L821
>
> if mutator is allocating and pacer enabled (default) we enter Pacer:
>
>
> https://github.com/openjdk/jdk/blame/739769c8fc4b496f08a92225a12d07414537b6c0/src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp#L828
>
>
> https://github.com/openjdk/jdk/blame/master/src/hotspot/share/gc/shenandoah/shenandoahPacer.cpp#L229
>
> and I assume try to handle it nicely, if not we start pacing:
>
>
> https://github.com/openjdk/jdk/blame/master/src/hotspot/share/gc/shenandoah/shenandoahPacer.cpp#L253
>
>
>
> I have few questions here:
>
> - Could you please explain a bit how the system of taxes works?  I assume
> mutators claim budget, while GC replenishes it async, but the details are
> missing and no comments in the code
>
> - To pace we use wait function from Monitor class
>
>
> https://github.com/openjdk/jdk/blame/master/src/hotspot/share/runtime/mutex.cpp#L232
>
>   but the first thing it gets current Java thread. Does that mean that
> each Java thread goes throw runtime -> heap to allocate, and that's how
> pacer paces it? So we just pace any allocating thread and threads that
> allocate more will just hit this code more often.
>
>
>
> - Pacer uses ShenandoahPacingMaxDelay (10ms) as max, but
> pace_for_allocation returns void
>
>
> https://github.com/openjdk/jdk/blob/fba763f82528d2825831a26b4ae4e090c602208f/src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp#L826
>
>
> https://github.com/openjdk/jdk/blame/master/src/hotspot/share/gc/shenandoah/shenandoahPacer.cpp#L225
>
> so I assume if there is no budget available it will pace a thread for up
> to 10ms, but it does not imply allocation failure.
>
> Heap class tries to allocate under lock and if unsuccessful considers this
> as allocation failure and handles it by calling ShenandoahControlThread.
> Does it mean that Pacer can’t cause GC to switch to degenerated mode or I
> am missing something?
>
>
>
> - If Pacer doesn't have budget to allocate memory it paces thread, but is
> there any global budget for pacing time or it is only per thread max
> (ShenandoahPacingMaxDelay)?
>
> - It would really nice if you can sched some light on these transitions:
> Concurrent Mode -> Pacing (single thread and total pacing time for all
> threads) -> most importantly logic of transitioning from pacing to
> degenerated GC
>
>
>
> I am trying to build a model which can tell me whether GC is healthy
> (fully concurrent), a bit unhealthy (pacing), unhealthy (degenerated or
> full GC) and how close are to the edge of the next state (a bit unhealthy
> -> unhealthy)
>
>
>
> No rush and thanks a lot in advance.
>
>
>
> Regards,
>
> Alex Dubrouski
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/shenandoah-dev/attachments/20221122/8250def8/attachment-0001.htm>