Shenandoah performance problem

[Attila Axt]

>> https://imgur.com/S2vEmnL
> I believe this shows that non-generational Shenandoah uses more CPU than a generational CMS: so the
> GC cycles are longer on average, and then the total CPU time is larger. For single-threaded
> workload, the heap occupancy might be much lower, so GC cycle length is smaller?
>
> It would be interesting to zoom in and see if CPU utilization differs when GC is idle to confirm.

Sorry, I wasn't clear enough about the measurement setup. All those 
graphs are measured the same time.

We are running a recommendation service. The incoming traffic is 
load-balanced between serveral application servers. I did the 
measurement on two of those servers, one configured with CMS, the other 
configured with Shenandoah.  These two nodes are receiving roughly the 
same traffic.

What you see on the graph is not the total cpu-time by various 
standalone test, but instead the cpu-time of a thread/threadgroup of 
this application.

I wrote a simple tool, which queries the cpu time usage of every 
thread(group) in every 5 seconds . If I display the results in a 
top-like manner, it looks like this:

Elapsed time, S:1260.29s, L:5.07s
     SCPU      SCPU/s       SCPU%        LCPU      LCPU/s LCPU%         
TID    NAME (COUNT)
  2871394     2278.36           -       12794 2521.72           
-                SUMMARY
  1038532      824.04       36.17        4257      839.06 33.27         
649    http-main-# (25)
   764557      606.65       26.63        3084      607.86 24.11       
18198    Process Queue (1)
   494317      392.22       17.22        3276      645.70 25.61       
17556    Shenandoah GC Threads## (6)
   152115      120.70        5.30         582      114.71 4.55       
17960    Xmemcached-Reactor-# (20)
   109960       87.25        3.83         434       85.54 3.39       
17584    Log4j2-TF-1-AsyncLogger (1)
   [...]

CPU: is the cpu time burned between the last measurement and the current 
measurement. The prefix 'S' means from the start of the tool , 'L' means 
from the last measurement 5 seconds ago.
CPU/s: CPU divided by the length of the measurement period, basically 
average cputime burned by the threadgroup
CPU%: percentage of this threadgroups cpu usage


As you can see on the above snapshot, the 'http-main-#', 'Process 
Queue', and 'Shenandoah GC Threads' are responsible ~80% of the cpu time 
burned, I named those on the graph as 'Multi threaded workload', 'Single 
threaded workload' and GC.

What you see on the graphs, is the average cpu time burned by those 
threadgroups with a 5 second resolution (LCPU/s column).

Cheers,
   axt