RFC: Draft JEP: Improved Metaspace allocator

[charlie hunt]

To add to Thomas Schatzl's comments ...

The stress test mentioned is something we would not be able to share 
since it looks like there may be some redistribution restrictions with 
some of its components.

A little while ago you posted a link to the following webrev:
http://cr.openjdk.java.net/~stuefe/webrevs/autouncommit-metachunks/webrev.00/webrev/index.html

A build with the above changes added to it and running the same stress 
test showed better behavior.

The above said, the stress test is rather extreme.

I also had not realized jcmd "VM.metaspace" existed. Thanks for the tip!

hths,

Charlie

On 3/27/19 11:01 AM, Thomas Schatzl wrote:
> Hi,
>
> On Wed, 2019-03-27 at 16:29 +0100, Thomas Stüfe wrote:
>> Hi Thomas,
>>
>> On Wed, Mar 27, 2019 at 2:57 PM Thomas Schatzl <
>> thomas.schatzl at oracle.com> wrote:
>>> On Wed, 2019-03-20 at 20:19 +0100, Thomas Stüfe wrote:
>>>> Dear all,
>>>>
>>>> I would like to hear what you think about the following proposal:
>>>>
>>>> https://bugs.openjdk.java.net/browse/JDK-8221173
>>>>
>>>> Do you consider this worthwhile?
>>> Yes, definitely!
>> Great :)
>>   
>>> At the beginning of this month we were internally looking at a
>>> metaspace stress application, and it took some time to nail the
>>> problem on metaspace fragmentation. Improving fragmentation, the
>>> visibility of such problems (fragmentation statistics) and
>>> potential options to e.g. favor compactness vs. performance.
>> Just to be sure, are you aware of the jcmd "VM.metaspace" command we
>> (RedHat & SAP) added for jdk11?
> No, at least I did not. At least not at the time when it would have
> been very useful :(
>
>> It can be used to analyze Metaspace in quite some detail. We did not
>> make a big fuss over it, but it is there and quite useful.
> Which is obviously the problem.
>
>> It may of course be missing functionality; if yes, please tell us.
> I will forward this information. I think we may get some time to do
> some more runs and look over the output.
>
>> About your stress test, I'd be interested in a reproduction case if
>> that is possible. I am always collecting pathological cases to play
>> with.
> I will bring that up too.
>
>>>> I debated with myself whether a JEP is the correct umbrella for
>>>> it. In the past I did larger changes like this without JEP but in
>>>> this case I feel better with a confirmed and sponsored JEP
>>>> behind me - from a certain size on it just feels more
>>>> appropriate.
>>>>
>>>> As usual, I am not sure where to put Metaspace-related proposals
>>>> to catch all interested people, so I start with hs-dev. Please
>>>> feel free to redirect if needed.
>>> You are right that a JEP is the only way to get guaranteed
>>> visibility to people. JEPs are mostly the only source for third
>>> party "what's new in JDK XX" write-ups.
>>> So if you want to reach a wide (non-hotspot-dev) audience, JEPs are
>>> basically the only way. I could kind of see that this might be the
>>> case.
>> For me it is more of a question of "I don't want to make a fuss" vs
>> "I don't want to sneak large changes in behind everyone's back". In
>> this case, I side with the latter.
>>
>> As additional advantage, a JEP gives a notion of progress to the
>> outside world. And I have the feeling it activates more Reviewer
>> muscle than large-non-JEP patches.
> :)
>
> Thanks,
>    Thomas
>