RFR(XS) 8191931: NMT test for anonymous/delegating metadata space sizing

Thomas Stüfe thomas.stuefe at gmail.com
Thu Mar 15 18:29:08 UTC 2018


Hi Zhengyu,

On Thu, Mar 15, 2018 at 7:15 PM, Zhengyu Gu <zgu at redhat.com> wrote:

> Hi Thomas,
>
> On 03/15/2018 01:52 PM, Thomas Stüfe wrote:
>
>> Hey Zhengyu,
>>
>> Sorry for that.
>>
>> I guess you could do statistical tests. Maybe java based jtreg tests,
>> generating a number of lambdas etc, then using NMT to print your statistics
>> and analysing it - requiring waste to be smaller than a certain percentage.
>>
>> That test would be valuable for catching regressions, but from my
>> experience take time to get right (to eliminate false positives etc).
>>
>
> Maybe. But I still do see that can be *reliable*, cause the code is
> generated, size may change over the time.
>
>
True. Not easy.


> Besides, there is no way to tell which are Lambdas, etc. from test, which
> are loaded from bootstrap class loaders, etc.
>
> Another obstacle is that, there is limited capability to capture output
> from jtreg test, other than ShouldContain/ShouldNotContain, etc. I am not
> aware of way to parse NMT output -- Do I miss some API here?
>

We (SAP) have a test library where we use regex to parse logs in a more
sophisticated way, including extracting numericals from logs etc. We should
contribute this some time. One could do all this manually, using java
regular expressions. But it is a lot of boilerplate coding.


> Thanks,
>
> -Zhengyu
>
>
Maybe in the end a simpler and much more practical test could be a test
case with a large number of parallel lambdas and a limiting
MaxMetaspaceSize. Maybe <what we need now> * 1.5 or 2?

If that test triggers the MaxMetaspaceSize it is always worth looking at,
because it means the metadata footprint increased for this scenario for
some reason, and a probable reason would be that someone broke your
optimization.

Thanks, Thomas



>
>
>
>> ... Thomas
>>
>>
>>
>> On Thu, Mar 15, 2018 at 4:37 PM, Zhengyu Gu <zgu at redhat.com <mailto:
>> zgu at redhat.com>> wrote:
>>
>>     Hi Thomas,
>>
>>     On 03/15/2018 11:13 AM, Thomas Stüfe wrote:
>>
>>         Hi Zhengyu,
>>
>>         http://cr.openjdk.java.net/~zgu/8191931/webrev.00/src/hotspo
>> t/share/memory/metaspace.cpp.udiff.html
>>         <http://cr.openjdk.java.net/~zgu/8191931/webrev.00/src/hotsp
>> ot/share/memory/metaspace.cpp.udiff.html>
>>
>>         Not sure, but would this test not break if the class loader
>>         allocates a metablock which does not fit into a specialized
>>         chunk before reaching
>>         _anon_and_delegating_metadata_specialize_chunk_limit - which is
>>         a theoretically valid scenario?
>>
>>         See calc_chunk_size():
>>
>>             if ((_space_type == Metaspace::AnonymousMetaspaceType ||
>>         _space_type == Metaspace::ReflectionMetaspaceType) &&
>>                 _mdtype == Metaspace::NonClassType &&
>>                 sum_count_in_chunks_in_use(SpecializedIndex) <
>>         _anon_and_delegating_metadata_specialize_chunk_limit &&
>>                 word_size + Metachunk::overhead() <= SpecializedChunk)
>>         {             <<<<
>>               return SpecializedChunk;
>>             }
>>
>>     Oops, I completely messed up here.
>>
>>     I guess there is *not* reliable way to test this, cause it can jump
>>     to any size in theory ...
>>
>>     Thanks,
>>
>>     -Zhengyu
>>
>>
>>
>>         In other words, encountering small/medium chunks before reaching
>>         the spec chunk limit could be, at least according to
>>         calc_chunk_size(), a valid albeit rare scenario.
>>
>>         --
>>
>>         nits:
>>
>>         "should use up to"  -> "should use at least"
>>         "midium" -> "medium"
>>
>>         Should the comment not be:
>>
>>         -// _anon_and_delegating_metadata_specialize_chunk_limit small
>>         chunks before jump to midium chunks,
>>         +// _anon_and_delegating_metadata_specialize_chunk_limit
>>         specialized chunks before jump to medium chunks,
>>
>>         ?
>>
>>         Do we always jump right to medium chunks from specialized chunks?
>>
>>         --
>>
>>         If your intention is to test "there shall be no small/medium
>>         chunks before we have reached the limit of specialized chunks",
>>         would it not be easier to test just that:
>>
>>         if (sum_count_in_chunks_in_use(MediumIndex) > 0) {
>>             assert(sum_count_in_chunks_in_use(SpecializedIndex) >= limit)
>>         }
>>
>>         instead of calculating and comparing the capacity?
>>
>>         Best Regards, Thomas
>>
>>
>>
>>         On Thu, Mar 15, 2018 at 3:52 PM, Zhengyu Gu <zgu at redhat.com
>>         <mailto:zgu at redhat.com> <mailto:zgu at redhat.com
>>         <mailto:zgu at redhat.com>>> wrote:
>>
>>              Hi,
>>
>>              Please review a new gtest to ensure correct sizing for
>>         anonymous and
>>              reflection delegating classloade's metadata space.
>>
>>              Bug: https://bugs.openjdk.java.net/browse/JDK-8191931
>>         <https://bugs.openjdk.java.net/browse/JDK-8191931>
>>              <https://bugs.openjdk.java.net/browse/JDK-8191931
>>         <https://bugs.openjdk.java.net/browse/JDK-8191931>>
>>              Webrev:
>>         http://cr.openjdk.java.net/~zgu/8191931/webrev.00/index.html
>>         <http://cr.openjdk.java.net/~zgu/8191931/webrev.00/index.html>
>>                     <http://cr.openjdk.java.net/~z
>> gu/8191931/webrev.00/index.html
>>         <http://cr.openjdk.java.net/~zgu/8191931/webrev.00/index.html>>
>>
>>
>>              Thanks,
>>
>>              -Zhengyu
>>
>>
>>
>>


More information about the hotspot-runtime-dev mailing list