Strange behaviour on half-occupied heap

[Aleksey Shipilev]

On 08/29/2018 04:06 PM, Simon Ogorodnik wrote:
> I think that homogenous fragmentation is not an origin of the problem, as it was only triggering to
> fall into Full GC
> 
> Here other similar situation, but with a whole heap full of ’Shared allocs’

That are quite probably the used space after previous Full GC. It resets all live data to "Shared"
after collection.

> According to HPROF heap dump before Full GC, there are 1.3 GB of 'Objects unreachable from GC roots,
> but not yet collected, Retained 1378197088’
> But GC didn’t bother to clear it before Full GC
> 
> After Full GC house objects actually got cleaned up
> 
> GCLogs & HPROF’s
> https://drive.google.com/file/d/1oNdTsRMcb_DBoBnTbkZgoxSqkaGkz6Oi/view?usp=sharing

Thanks! Heap dumps do not yield any suspicion. There are lots of int[] arrays from BufferedImage's,
which I would expect from IDE workload.

> Adaptive CSet Selection. Target Free: 204M, Actual Free: 0M, Max CSet: 0M, Min Garbage: 204M
> 
> Looks like it size of collectable garbage is wrong, because of according to heap dump there are
> actually 1.3G of not yet collected garbage 
> 
> Collectable Garbage: 0M (0% of total), 0M CSet, 0 CSet regions 
> Immediate Garbage: 0M (0% of total), 0 regions

This interpretation is incorrect: there is 204M of free space, which means there is ~1.8 GB used.
Out of those 1.8 GB, some part is real garbage. We don't know which part (we can add the printing
for it, by the way). "Collectable garbage" means the garbage in the collection set, but the
collection set is 0 regions in this cycle, so collectable garbage is also 0.

Full GC compacts everything (this is why it is "full"), so it reclaims a lot.

The real question is how did we get into the situation there is an empty collection set. I'll study
the logs for that. What you can find by grepping for "Trigger:" is that "Free" gradually decreases
like this, until it reaches Full GC:

Trigger: Free (160M) is below minimum threshold (204M)
Trigger: Free (131M) is below minimum threshold (204M)
Trigger: Free (131M) is below minimum threshold (204M)
Trigger: Free (122M) is below minimum threshold (204M)
Trigger: Free (121M) is below minimum threshold (204M)

I think I saw this recently, and the fix was this:
  http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-August/007218.html

I have to make sure it works reliably everywhere, and then push it. Meanwhile, you can try set
-XX:ShenandoahMaxCSetFactor=60, and watch the "Trigger" lines. Or, give it more headroom in heap.

-Aleksey