G1 native memory consumption

[Milan Mimica]

Hi Thomas

Thanks for your input. I took me a while to have a stable system again to repeat measurements.

I have tried setting G1HeapRegionSize to 16M on one instance (8M is default) and I notice lower GC memory usage:
GC (reserved=1117MB -18MB, committed=1117MB -18MB)
vs
GC (reserved=1604MB +313MB, committed=1604MB +313MB)

It seems more stable too. However, "Internal" is still relatively high for a 25G heap, and there is no much difference between instances:
Internal (reserved=2132MB -7MB, committed=2132MB -7MB)




Milan Mimica, Senior Software Engineer / Division Lead
________________________________________
From: Thomas Schatzl <thomas.schatzl at oracle.com>
Sent: Monday, January 23, 2017 11:41
To: Milan Mimica; hotspot-gc-use at openjdk.java.net
Subject: Re: G1 native memory consumption

Hi Milan,

On Fri, 2017-01-20 at 20:24 +0000, Milan Mimica wrote:
> Hi
>
> I'm inspecting memory consumption issues of a service running on
> java-8u102, linux. The service is running for a few days now, and in
> a few days more it would consume all of 32GB physical memory
> available, and get killed by OOM Killer.
> Questions:
> - If the code is not allocating any significant off-heap memory,
> neither by Unsafe.allocateMemory or by external library, isn't 7GB
> native memory overhead supposed to be enough for a 25GB heap?
> - Why so much memory spent on "Internal" category, apparently from G1
> thread?

G1 remembered sets typically consume approximately 10% of java heap,
depending on application, heap and your remembered set configuration.

This remembered set contains information that is necessary to be able
to do incremental and partial old generation compaction.

So the 7GB should be sufficient.

You can decrease remembered set overhead by e.g. increasing heap region
size. Given the heap size you mentioned it seems it would be worth a
try to go to 16M regions.


> Find the attached jemalloc heap profile, showing "live"
> allocations that happened in about 30 hour timespan, and a
> NMT profile of approximately same period.
> Profiling was done after some warm-up time time, and with a
> manually triggered Full GC in between just to give the JVM a chance
> to clean up everything.

Remembered set memory consumption should level out after some time,
where "some" may be quite a bit of time. I.e. it may take much longer
than for other collectors. It also depends on the memory allocator.

Do you have any new measurements after this weekend? It should have
levelled out by this time.

JDK9 contains some improvements on memory usage in exactly this area.
There will likely be further improvements in this area going forward.

Thanks,
  Thomas