RFR (S): 8195115: G1 Old Gen MemoryPool CollectionUsage.used values don't reflect mixed GC results

[Hohensee, Paul]

We don’t use getType, and you guessed correctly: we use the memory pool name as an indicator of the specific characteristics of a memory pool, in particular eden.

What we want is an indication of long term heap occupancy. We calculate it using CollectionUsage for non-eden heap memory pools, regardless of collector. We don’t use JMX notification, rather we periodically poll CollectionUsage for memory pools with names that contain “Old”, “Tenured”, or “Survivor”. We get the memory pools from the GarbageCollectorMXBeans (we don’t care what the collector names are). For the named memory pools, we sum CollectionUsage.used and divide by the sum of CollectionUsage.max to get a long term heap occupancy percentage. We don’t want to include eden because it’s really just an allocation buffer and not part of the storage for long-lived objects. I suppose we could use a negative test instead by using all memory pools with names that don’t include “Eden”.

The bug is that the “G1 Old Gen” memory pool isn’t being updated when the “G1 Young Generation” collector runs a mixed collection. As far as JMX is concerned, that collector only knows about eden and the survivor space. The patch adds the old gen to the memory pools it knows about and has mixed collections update the old gen’s CollectionUsage.

I managed to get a submit repo run to succeed last week and it found a problem. I’ve uploaded a new webrev that fixes the failure of the jtreg test TestMemoryMXBeansAndPoolsPresence.java due to the young gen collector being expected to know only about eden and the survivor space.

http://cr.openjdk.java.net/~phh/8195115/webrev.hs.01/

Waiting on the submit repo to come back with a result on it.

Thanks,

Paul

From: mandy chung <mandy.chung at oracle.com>
Organization: Oracle Corporation
Date: Monday, January 29, 2018 at 10:52 AM
To: "Hohensee, Paul" <hohensee at amazon.com>, Erik Helin <erik.helin at oracle.com>, David Holmes <david.holmes at oracle.com>
Cc: "serviceability-dev at openjdk.java.net" <serviceability-dev at openjdk.java.net>, "hotspot-gc-dev at openjdk.java.net" <hotspot-gc-dev at openjdk.java.net>
Subject: Re: RFR (S): 8195115: G1 Old Gen MemoryPool CollectionUsage.used values don't reflect mixed GC results


On 1/29/18 10:35 AM, mandy chung wrote:
Thanks for the reply Paul.   Try to understand a little more on the specific from gc-specific memory pool you depend on.
On 1/29/18 8:27 AM, Hohensee, Paul wrote:
A name change would affect Amazon’s heap monitoring, and thus I expect it would affect other users as well.

As long as there are gc-specific memory pools, we’re going to need to be able to identify them, and right now that’s done via name.

MemoryPoolMXBean::getType returns "heap" memory type for GC-specific memory pools.  Are you using this method?  Do you use the name to build in specific characteristic of a memory pool (e.g. eden vs old gen)?



All the mxbeans are identified by name, so that’s a general design principle. The only way I can think of to get rid of name dependency would be to figure out what abstract metrics users want to monitor and implement them for all collectors. HeapUsage (instantaneous occupancy) is one, CollectionUsage (long-lived occupancy) is another, both of these for the entire heap, not just particular memory pools.

The sum of HeapUsage and CollectionUsage of all heap memory pools was expected to give an incorrect approximation for the entire heap usage.  Are you seeing issue/bug with the sum result?

typo: s/an incorrect approximation/an approximation.

Mandy


Mandy


That said, imo there will always be a demand for the ability to get collector and memory pool specific details, so I don’t see a way to get around providing named entities.

Paul

From: mandy chung <mandy.chung at oracle.com><mailto:mandy.chung at oracle.com>
Organization: Oracle Corporation
Date: Friday, January 26, 2018 at 2:38 PM
To: "Hohensee, Paul" <hohensee at amazon.com><mailto:hohensee at amazon.com>, Erik Helin <erik.helin at oracle.com><mailto:erik.helin at oracle.com>, David Holmes <david.holmes at oracle.com><mailto:david.holmes at oracle.com>
Cc: "serviceability-dev at openjdk.java.net"<mailto:serviceability-dev at openjdk.java.net> <serviceability-dev at openjdk.java.net><mailto:serviceability-dev at openjdk.java.net>, "hotspot-gc-dev at openjdk.java.net"<mailto:hotspot-gc-dev at openjdk.java.net> <hotspot-gc-dev at openjdk.java.net><mailto:hotspot-gc-dev at openjdk.java.net>
Subject: Re: RFR (S): 8195115: G1 Old Gen MemoryPool CollectionUsage.used values don't reflect mixed GC results


On 1/25/18 1:04 PM, Hohensee, Paul wrote:

> The JMX API spec doesn’t specify what the memory pool or garbage > collector names are, but the current names are de-facto part of the > API, so if we change the existing ones, imo a CSR should be filed.

The names are implementation details but I can see how an application

might be impacted if they depend on it.  CSR approval is not strictly

necessary while I think filing one to document the change would be

good.


Does the name change impact any application you know of?  I'm trying to

understand if any improvement to API is needed so that applications

don't need to depend on the names.

Mandy





-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20180129/28199100/attachment-0001.html>