RFR(s): 8189864: Provide an ascii map to visualize metaspace fragmentation

[Thomas Stüfe]

Oh, forgot mention:

Goetz and I discussed offlist his idea of adding in-chunk occupation to the
map. The idea is good, but we decided to keep this for a separate RFE later
and go with the current patch now as it is.

Thanks!

Thomas



On Wed, Nov 8, 2017 at 5:21 PM, Thomas Stüfe <thomas.stuefe at gmail.com>
wrote:

> Hi Coleen, Goetz,
>
> new webrev, rebased to the new jdk-hs repo.
>
> http://cr.openjdk.java.net/~stuefe/webrevs/8189864-
> metaspace-map/webrev.01/webrev/
>
> Nothing changed.
>
> @Coleen, would you be able to sponsor this? Thanks a lot!
>
> Kind Regards, Thomas
>
> On Wed, Nov 1, 2017 at 8:20 AM, Thomas Stüfe <thomas.stuefe at gmail.com>
> wrote:
>
>> Hi Goetz,
>>
>> On Mon, Oct 30, 2017 at 1:49 PM, Lindenmaier, Goetz <
>> goetz.lindenmaier at sap.com> wrote:
>>
>>> Hi Thomas,
>>>
>>> the change looks good and I know we have made good use of this
>>> already.
>>>
>>> Can you look into the chunks? It could be useful to know that,
>>> for example, a medium chunk is used by a  class loader, but
>>> still mostly empty (i.e., empty but not free).  Well, there is no
>>> third variant after lower and upper case,  but maybe empty
>>> parts could be indicated in the line with the dots by 'e's (obviously
>>> at the granularity of small chunks).
>>>
>>> Thanks,
>>>   Goetz.
>>>
>>>
>> thank you for the review!
>>
>> Indicating which chunks are filled to what degree can certainly be done.
>> Question is how useful this is (see Zhengyu's work for
>> https://bugs.openjdk.java.net/browse/JDK-8189688), because within chunks
>> there is no fragmentation, so the in-chunk map could look rather
>> unexciting. I will take a look when I am back next week.
>>
>> Kind Regards, Thomas
>>
>> > -----Original Message-----
>>> > From: hotspot-runtime-dev [mailto:hotspot-runtime-dev-
>>> > bounces at openjdk.java.net] On Behalf Of Thomas Stüfe
>>> > Sent: Wednesday, October 25, 2017 6:52 AM
>>> > To: hotspot-runtime-dev at openjdk.java.net
>>> > Subject: RFR(s): 8189864: Provide an ascii map to visualize metaspace
>>> > fragmentation
>>> >
>>> > Hi all,
>>> >
>>> > could I please have your thoughts and reviews for this enhancement:
>>> >
>>> > Issue:  https://bugs.openjdk.java.net/browse/JDK-8189864
>>> > Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8189864-
>>> > metaspace-map/webrev.00/webrev/
>>> >
>>> > At SAP, we added a something we call a metaspace map to the metaspace
>>> > analysis functions. This is a very simple ASCII print of the metaspace
>>> > layout. It shows the composition of the VirtualSpaceNodes on a chunk
>>> basis.
>>> > It facilitates examining fragmentation and has been proven useful when
>>> > experimenting with the metaspace allocator.
>>> >
>>> > This change adds the map printing code and invokes it if a Metaspace
>>> OOM
>>> > occurs and -Xlog:gc+metaspace+freelist=debug is active (in this case,
>>> we
>>> > already print out a bunch of things). We also may consider adding this
>>> to
>>> > NMT or as a jcmd addition, but I did not want to overload the patch.
>>> >
>>> > Example output looks like this (mind that this will only make sense
>>> with a
>>> > monospaced font). Dots indicate starts of chunks. Letters indicate
>>> chunk
>>> > type - (H)umongous, (M)edium, (S)mall (X)specialized - with lower case
>>> > letters indicating
>>> > a free chunk, upper case letters indicating a chunk in use.
>>> >
>>> > 0x0000000100000000:   ......
>>> >                       xxxxxxHHHHHHHHHHHHHHHHHHHHHHHH
>>> > HHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH
>>> > HHHHHHHHH
>>> > HHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH
>>> > 0x0000000100020000:
>>> >                       HHHHHHHHHHHHHHHHHHHHHHHHHHHHHH
>>> > HHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH
>>> > HHHHHHHHH
>>> > HHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH
>>> > 0x0000000100040000:
>>> >                       HHHHHHHHHHHHHHHHHHHHHHHHHHHHHH
>>> > HHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH
>>> > HHHHHHHHH
>>> > HHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH
>>> > 0x0000000100060000:   . . . . . . . . . . . . . .
>>> >                       SSSSSSSSSSSSMMMMMMMMMMMMMMMMMM
>>> > MMMMMMMMMMMMMMSSSSSSSSMMMMMMMMMMMMMMMMMMM
>>> > MMMMMMMMMMMMMMMMMMM
>>> > MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
>>> > M
>>> > 0x0000000100080000:   . . . .
>>> >                       MMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
>>> > MMMMMMMMMMMMMMMMMMMMMMmmmmmmmmmmmmmmmm
>>> > mmmmmmmmmmmmmmmmMMMMMM
>>> > MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
>>> > M
>>> > 0x00000001000a0000:   . . . .
>>> >                       MMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
>>> > MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
>>> > MMMMMMMMMMMMMMMMMMMMMMM
>>> > MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
>>> > M
>>> > 0x00000001000c0000:   . . . .
>>> >                       MMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
>>> > MMMMMMMMMMMMMMMMMMMMMMmmmmmmmmmmmmmmmm
>>> > mmmmmmmmmmmmmmmmMMMMMM
>>> > MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
>>> > M
>>> > 0x00000001000e0000:   . . . .
>>> >                       MMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
>>> > MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
>>> > MMMMMMMMMMMMMMMMMMMMMMM
>>> > MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
>>> > M
>>> > 0x0000000100100000:   . . . . . . . . . . . . . . . . . . . . . . . .
>>> >                       MMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
>>> > MMMMMMMMMMMMMMMMMMMMMMSSSSMMMMMMMMMMMMM
>>> > MMMMMMMMMMMMMMMMMMMSS
>>> > SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
>>> > 0x0000000100120000:   . . . . . . . . . . . . . . . . . . . . . . . .
>>> . . .
>>> > . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
>>> . .
>>> >                       SSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
>>> > SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
>>> > SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
>>> >
>>> >
>>> > Thank you!
>>> >
>>> > Thomas
>>>
>>
>>
>