RFR(S) 8189688: NMT: Report per-class load metadata information

Zhengyu Gu zgu at redhat.com
Mon Oct 23 18:19:49 UTC 2017


Hi Thomas,

I added chunkmanager statistics to the output.

However, I did not revive per-classload summary line. Had Second 
thought, that NMT probably should just report facts and leave 
interpretation to users.

Updated webrev: http://cr.openjdk.java.net/~zgu/8189688/webrev.01/index.html

Thanks,

-Zhengyu


On 10/23/2017 11:08 AM, Thomas Stüfe wrote:
> 
> 
> On Mon, Oct 23, 2017 at 5:03 PM, Zhengyu Gu <zgu at redhat.com 
> <mailto:zgu at redhat.com>> wrote:
> 
> 
> 
>         Okay. So this is important for understanding cases where we have
>         a large number of miniscule class loaders, each one only having
>         a few numbers of chunks. In this case, would it be useful to
>         have a running total of "free" and "waste", too? Or even better,
>         also average and peak values of "free" and "waste" (to tell
>         apart cases where you have one outlier vs cases where just all
>         metaspaces waste a lot of memory).
>         Just out of curiosity, I looked at
>         http://cr.openjdk.java.net/~zgu/cld_metaspace/wildfly.txt
>         <http://cr.openjdk.java.net/~zgu/cld_metaspace/wildfly.txt>  and
>         it seems that "free" is the problem, not "waste"? So I guess
>         what happens is that all the little classloaders allocate just
>         enough memory to push them over "_small_chunk_limit=4", so they
>         allocate the first medium chunk, from which then they take a
>         small bite and leave the rest laying around?
> 
>     Yes. The free space of anonymous classes should be counted as waste,
>     in my opinion. I am not sure if everyone agrees, so I took the
>     summary line out of this patch.
> 
>     I would be more than happy to restore the summary line, if you find
>     it is useful :-)
> 
> 
> I agree with you. Typically class loaders stop loading at some point, 
> especially these tine ones, and often will not resume loading. So, 
> effectivly, the space is wasted. It still helps to distinguish "free" in 
> the current chunks from "free" in the other chunks to tell this 
> situation apart from a situation where we have just a bug in metaspace 
> chunk handling preventing us to use up our chunks.
> 
> 
>                  The latter would be an important addition, regardless
>         if this is
>                  for customers or for us. Chunks idling in freelists can
>         make a
>                  huge impact, and unfortunately this may happen in perfectly
>                  valid cases. See e.g. our JEP
>                  "https://bugs.openjdk.java.net/browse/JDK-8166690
>         <https://bugs.openjdk.java.net/browse/JDK-8166690>
>                  <https://bugs.openjdk.java.net/browse/JDK-8166690
>         <https://bugs.openjdk.java.net/browse/JDK-8166690>>". We have
>                  already a printing routine for free chunks, in
>                  ChunkManager::print_all_chunkmanagers(). Could you add
>         this to
>                  your output?
> 
> 
>              Make sense! Could you recommend what data and format you
>         would like
>              to see?
> 
> 
> 
>         Would not ChunkManager::print_all_chunkmanagers() be sufficient?
> 
>     Okay, I might need to tweak output format.
> 
> 
> Fine by me. Nothing depends on it yet.
> 
> Thanks, Thomas
> 
>     Thanks,
> 
>     -Zhengyu
> 
> 
> 
>                  -------------
> 
>                  Other than above notes, the changes seem
>         straighforward, I did
>                  not see anything wrong. Minor nits:
> 
>                  - PrintCLDMetaspaceInfoClosure::_out, _scale and _unit
>         could all
>                  be constant (with _out being a outputStream* const).
>                  - We also do not need to provide scale *and* unit as
>         argument,
>                  one can be deduced from the other. This would also prevent
>                  mismatches.We do need scale here, since nmt command
>         line can set
>                  scale and we do
> 
>              have to honor that.
> 
>              However, passing unit is ugly! I don't want to introduce
>         inverse
>              dependence (metaspace -> nmt), which is also ugly.
> 
> 
>         Yes, that would be regrettable.
> 
>              Probably should move scale mapping code from NMTUtil to a
>         common util?
> 
> 
> 
>         That would be possible, these functions (scale_from_name etc)
>         are simple enough to be moved to a generic file. However, if you
>         pass scale, deducing the string that goes with it is trivial and
>         I would have no qualms doing this by hand in metaspace.cpp. But
>         it is a matter of taste.
> 
> 
>                  I did not look closely at locking issues, I assume
>                  MetaspaceAux::print_metadata() is supposed to run only in
>                  Safepoints?
> 
> 
>              No. sum_xxx_xxx_in_use queries are guarded by locks.
> 
>              Thanks,
> 
>              -Zhengyu
> 
> 
>         Thanks, Thomas
> 
> 
> 
>                  Thanks & Kind Regards, Thomas
> 
> 
> 
> 
> 
> 
>                  On Fri, Oct 20, 2017 at 4:00 PM, Zhengyu Gu
>         <zgu at redhat.com <mailto:zgu at redhat.com>
>                  <mailto:zgu at redhat.com <mailto:zgu at redhat.com>>
>         <mailto:zgu at redhat.com <mailto:zgu at redhat.com>
> 
>                  <mailto:zgu at redhat.com <mailto:zgu at redhat.com>>>> wrote:
> 
>                       Up to now, there is little visibility into how
>         metaspaces
>                  are used,
>                       and by whom.
> 
>                       This proposed enhancement gives NMT the ability to
>         report
>                  metaspace
>                       usages on per-classloader level, via a new NMT command
>                  "metadata"
> 
> 
>                       For example:
> 
>                       2496:
>                       Metaspaces:
>                          Metadata space: reserved=      8192KB
>         committed=      5888KB
>                          Class    space: reserved=   1048576KB
>         committed=       640KB
> 
>                       Per-classloader metadata:
> 
>                       ClassLoader: for anonymous class
>                          Metadata   capacity=      5.00KB used=      1.16KB
>                  free=         3.84KB waste=      0.05KB
>                          Class data capacity=      1.00KB used=      0.59KB
>                  free=         0.41KB waste=      0.00KB
> 
>                       ....
> 
>                       ClassLoader: <bootloader>
>                          Metadata   capacity=   5640.00KB used=   5607.42KB
>                  free=         32.58KB waste=      0.46KB
>                          Class data capacity=    520.00KB used=    500.94KB
>                  free=         19.06KB waste=      0.02KB
> 
> 
>                       Bug:
>         https://bugs.openjdk.java.net/browse/JDK-8189688
>         <https://bugs.openjdk.java.net/browse/JDK-8189688>
>                  <https://bugs.openjdk.java.net/browse/JDK-8189688
>         <https://bugs.openjdk.java.net/browse/JDK-8189688>>
>                       <https://bugs.openjdk.java.net/browse/JDK-8189688
>         <https://bugs.openjdk.java.net/browse/JDK-8189688>
>                  <https://bugs.openjdk.java.net/browse/JDK-8189688
>         <https://bugs.openjdk.java.net/browse/JDK-8189688>>>
>                       Webrev:
>         http://cr.openjdk.java.net/~zgu/8189688/webrev.00/index.html
>         <http://cr.openjdk.java.net/~zgu/8189688/webrev.00/index.html>
>                 
>         <http://cr.openjdk.java.net/~zgu/8189688/webrev.00/index.html
>         <http://cr.openjdk.java.net/~zgu/8189688/webrev.00/index.html>>
>                             
>         <http://cr.openjdk.java.net/~zgu/8189688/webrev.00/index.html
>         <http://cr.openjdk.java.net/~zgu/8189688/webrev.00/index.html>
>                 
>         <http://cr.openjdk.java.net/~zgu/8189688/webrev.00/index.html
>         <http://cr.openjdk.java.net/~zgu/8189688/webrev.00/index.html>>>
> 
>                       Test:
> 
>                          hotspot_tier1_runtime, fastdebug and release on
>         x64 Linux.
> 
>                       Thanks,
> 
>                       -Zhengyu
> 
> 
> 
> 
> 


More information about the hotspot-runtime-dev mailing list