RFR(S) 8189688: NMT: Report per-class load metadata information

Thomas Stüfe thomas.stuefe at gmail.com
Mon Oct 23 15:08:40 UTC 2017


On Mon, Oct 23, 2017 at 5:03 PM, Zhengyu Gu <zgu at redhat.com> wrote:

>
>>
>> Okay. So this is important for understanding cases where we have a large
>> number of miniscule class loaders, each one only having a few numbers of
>> chunks. In this case, would it be useful to have a running total of "free"
>> and "waste", too? Or even better, also average and peak values of "free"
>> and "waste" (to tell apart cases where you have one outlier vs cases where
>> just all metaspaces waste a lot of memory).
>> Just out of curiosity, I looked at http://cr.openjdk.java.net/~zg
>> u/cld_metaspace/wildfly.txt  and it seems that "free" is the problem,
>> not "waste"? So I guess what happens is that all the little classloaders
>> allocate just enough memory to push them over "_small_chunk_limit=4", so
>> they allocate the first medium chunk, from which then they take a small
>> bite and leave the rest laying around?
>>
>> Yes. The free space of anonymous classes should be counted as waste, in
> my opinion. I am not sure if everyone agrees, so I took the summary line
> out of this patch.
>
> I would be more than happy to restore the summary line, if you find it is
> useful :-)
>
>
I agree with you. Typically class loaders stop loading at some point,
especially these tine ones, and often will not resume loading. So,
effectivly, the space is wasted. It still helps to distinguish "free" in
the current chunks from "free" in the other chunks to tell this situation
apart from a situation where we have just a bug in metaspace chunk handling
preventing us to use up our chunks.


>
>         The latter would be an important addition, regardless if this is
>>         for customers or for us. Chunks idling in freelists can make a
>>         huge impact, and unfortunately this may happen in perfectly
>>         valid cases. See e.g. our JEP
>>         "https://bugs.openjdk.java.net/browse/JDK-8166690
>>         <https://bugs.openjdk.java.net/browse/JDK-8166690>". We have
>>         already a printing routine for free chunks, in
>>         ChunkManager::print_all_chunkmanagers(). Could you add this to
>>         your output?
>>
>>
>>     Make sense! Could you recommend what data and format you would like
>>     to see?
>>
>>
>>
>> Would not ChunkManager::print_all_chunkmanagers() be sufficient?
>>
> Okay, I might need to tweak output format.
>
>
Fine by me. Nothing depends on it yet.

Thanks, Thomas


> Thanks,
>
> -Zhengyu
>
>
>>
>>         -------------
>>
>>         Other than above notes, the changes seem straighforward, I did
>>         not see anything wrong. Minor nits:
>>
>>         - PrintCLDMetaspaceInfoClosure::_out, _scale and _unit could all
>>         be constant (with _out being a outputStream* const).
>>         - We also do not need to provide scale *and* unit as argument,
>>         one can be deduced from the other. This would also prevent
>>         mismatches.We do need scale here, since nmt command line can set
>>         scale and we do
>>
>>     have to honor that.
>>
>>     However, passing unit is ugly! I don't want to introduce inverse
>>     dependence (metaspace -> nmt), which is also ugly.
>>
>>
>> Yes, that would be regrettable.
>>
>>     Probably should move scale mapping code from NMTUtil to a common util?
>>
>>
>>
>> That would be possible, these functions (scale_from_name etc) are simple
>> enough to be moved to a generic file. However, if you pass scale, deducing
>> the string that goes with it is trivial and I would have no qualms doing
>> this by hand in metaspace.cpp. But it is a matter of taste.
>>
>>
>>         I did not look closely at locking issues, I assume
>>         MetaspaceAux::print_metadata() is supposed to run only in
>>         Safepoints?
>>
>>
>>     No. sum_xxx_xxx_in_use queries are guarded by locks.
>>
>>     Thanks,
>>
>>     -Zhengyu
>>
>>
>> Thanks, Thomas
>>
>>
>>
>>         Thanks & Kind Regards, Thomas
>>
>>
>>
>>
>>
>>
>>         On Fri, Oct 20, 2017 at 4:00 PM, Zhengyu Gu <zgu at redhat.com
>>         <mailto:zgu at redhat.com> <mailto:zgu at redhat.com
>>
>>         <mailto:zgu at redhat.com>>> wrote:
>>
>>              Up to now, there is little visibility into how metaspaces
>>         are used,
>>              and by whom.
>>
>>              This proposed enhancement gives NMT the ability to report
>>         metaspace
>>              usages on per-classloader level, via a new NMT command
>>         "metadata"
>>
>>
>>              For example:
>>
>>              2496:
>>              Metaspaces:
>>                 Metadata space: reserved=      8192KB committed=
>> 5888KB
>>                 Class    space: reserved=   1048576KB committed=
>>  640KB
>>
>>              Per-classloader metadata:
>>
>>              ClassLoader: for anonymous class
>>                 Metadata   capacity=      5.00KB used=      1.16KB
>>         free=         3.84KB waste=      0.05KB
>>                 Class data capacity=      1.00KB used=      0.59KB
>>         free=         0.41KB waste=      0.00KB
>>
>>              ....
>>
>>              ClassLoader: <bootloader>
>>                 Metadata   capacity=   5640.00KB used=   5607.42KB
>>         free=         32.58KB waste=      0.46KB
>>                 Class data capacity=    520.00KB used=    500.94KB
>>         free=         19.06KB waste=      0.02KB
>>
>>
>>              Bug: https://bugs.openjdk.java.net/browse/JDK-8189688
>>         <https://bugs.openjdk.java.net/browse/JDK-8189688>
>>              <https://bugs.openjdk.java.net/browse/JDK-8189688
>>         <https://bugs.openjdk.java.net/browse/JDK-8189688>>
>>              Webrev:
>>         http://cr.openjdk.java.net/~zgu/8189688/webrev.00/index.html
>>         <http://cr.openjdk.java.net/~zgu/8189688/webrev.00/index.html>
>>                     <http://cr.openjdk.java.net/~z
>> gu/8189688/webrev.00/index.html
>>         <http://cr.openjdk.java.net/~zgu/8189688/webrev.00/index.html>>
>>
>>              Test:
>>
>>                 hotspot_tier1_runtime, fastdebug and release on x64 Linux.
>>
>>              Thanks,
>>
>>              -Zhengyu
>>
>>
>>
>>
>>


More information about the hotspot-runtime-dev mailing list