RFR(XS): 8193521: glibc wastes memory with default configuration

[Doerr, Martin]

Hi Andrew,

that's correct. The mmaps I have observed affect virtual memory. Not all of the memory gets committed.
So there are basically 2 issues:
- virtual memory: Gets somewhat larger than needed. May be an issue for users with reduced ulimit, cloud applications in containers, embedded.
- physical memory: We're not wasting so much, but if the JVM handles all performance critical allocations by its own management, it should be worth saving.

Related to your earlier mail:
> This space may be useless to the JVM code but it may not be useless to
> other native code. If would be good if you could substantiate your claim
> that this memory really is 'useless' (to all JVM users). Will your
> proposed change damage performance for some users? If so then can we
> scope the potential for damage and agree it is acceptable?

We are testing the change and didn't see performance regressions, yet. But we have to run more benchmarks, especially with G1 (see mail from Thomas Schatzl). Large server tests will be interesting as well.

I can't really estimate the risk for native libs and I can understand the concern. Maybe it would be acceptable if the change gets switchable.

Best regards,
Martin


-----Original Message-----
From: Andrew Dinn [mailto:adinn at redhat.com] 
Sent: Freitag, 15. Dezember 2017 12:38
To: Doerr, Martin <martin.doerr at sap.com>; Andrew Haley <aph at redhat.com>; hotspot-runtime-dev at openjdk.java.net
Subject: Re: RFR(XS): 8193521: glibc wastes memory with default configuration

On 15/12/17 10:39, Doerr, Martin wrote:
> The VM typically starts more than 20 threads even when only using
> java -version so we mmap several useless GB.

Also, I forgot to mention that it is not mmap calls per se that we need
to worry about but actual /physical commit/ of pages in the mmapped
region. We are not short of mappable address space and we don't incur
any significant cost by mapping the virtual address space we currently use.

I have no doubt your trace is showing a vmem page reservation rather
than a corresponding physical page commit. If the TLS regions mmapped in
your trace were really occupying several GBs of physical pages we would
already have done something about it.

regards,


Andrew Dinn
-----------
Senior Principal Software Engineer
Red Hat UK Ltd
Registered in England and Wales under Company Registration No. 03798903
Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander