RFR(S) 8186770: NMT: Report metadata information in NMT summary

[Andrew Dinn]

On 28/08/17 22:19, Zhengyu Gu wrote:
> This enhancement allows NMT to report class metadata information.
> 
> NMT has no visibility into metaspace so far, it has become an obstacle
> to estimate real memory cost for classes. While estimating the cost, we
> usually assume that class metadata occupies whole committed space, which
> results higher than actual number.

Of course, it's very important to know the committed size because that's
what you have to provision for. However, any disparity between that size
and the space actually occupied by metadata is also highly valuable
information. That's true whether the disparity arises because of waste
caused by fragmentation or because chunks have been added back to the
free list or have not yet been carved off recently malloced regions. So,
I think this extra info is very helpful.

> The patch uses existing metaspace APIs, and reports counters in NMT
> *Class* summary section.
> . . .
> Sample outputs:
> 
> Class summary:
> 
> -                     Class (reserved=1071790KB, committed=24750KB)
>                             (classes #3078)
>                             (malloc=686KB #7122)
>                             (mmap: reserved=1071104KB, committed=24064KB)
>                             (  Metadata:                               )
>                             (    reserved=22528KB, committed=21504KB)
>                             (    capacity=21327KB, used=20654KB)
>                             (    free chunks=113KB)
>                             (    available=0KB)
>                             (  Class space:                            )
>                             (    reserved=1048576KB, committed=2560KB)
>                             (    capacity=2525KB, used=2268KB)
>                             (    free chunks=0KB)
>                             (    available=35KB)
I think this change is to ship modulo a few small quibbles.

The four figures quoted here for each of the data and class metaspace
regions don't quite add up i.e.

  used + free chunks + available = total_in_use =/= capacity

  2268 + 0 + 35    =  2303   =/=  2525

  20654 + 113 + 0  =  20767  =/=  21327


As I understand it this is because of waste caused either by the need to
insert block and chunk headers or by the inability to allocate objects
out of small fragments at the end of in use chunks. Is that correct?

If so then would it not be clearer to account for this waste explicitly?
e.g.

                            (  Metadata:                            )
                            (    reserved=22528KB,   committed=21504KB)
                            (    capacity=21327KB,   used=20654KB)
                            (    free chunks=113KB,  available=0KB)
                            (    waste = 560KB = 2.6%)

n.b. the above figures are calculated as

  waste = capacity - total_in_use
  waste% = waste / capacity

Also, whether or not waste space gets reported, I think the output would
look cleaner if you were to report free and available space on one line.

I'm happ

regards,


Andrew Dinn
-----------
Senior Principal Software Engineer
Red Hat UK Ltd
Registered in England and Wales under Company Registration No. 03798903
Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander