RFR(XS) 8191931: NMT test for anonymous/delegating metadata space sizing

[Zhengyu Gu]

Hi Thomas,

On 03/15/2018 01:52 PM, Thomas Stüfe wrote:
> Hey Zhengyu,
> 
> Sorry for that.
> 
> I guess you could do statistical tests. Maybe java based jtreg tests, 
> generating a number of lambdas etc, then using NMT to print your 
> statistics and analysing it - requiring waste to be smaller than a 
> certain percentage.
> 
> That test would be valuable for catching regressions, but from my 
> experience take time to get right (to eliminate false positives etc).

Maybe. But I still do see that can be *reliable*, cause the code is 
generated, size may change over the time.

Besides, there is no way to tell which are Lambdas, etc. from test, 
which are loaded from bootstrap class loaders, etc.

Another obstacle is that, there is limited capability to capture output 
from jtreg test, other than ShouldContain/ShouldNotContain, etc. I am 
not aware of way to parse NMT output -- Do I miss some API here?

Thanks,

-Zhengyu



> 
> ... Thomas
> 
> 
> On Thu, Mar 15, 2018 at 4:37 PM, Zhengyu Gu <zgu at redhat.com 
> <mailto:zgu at redhat.com>> wrote:
> 
>     Hi Thomas,
> 
>     On 03/15/2018 11:13 AM, Thomas Stüfe wrote:
> 
>         Hi Zhengyu,
> 
>         http://cr.openjdk.java.net/~zgu/8191931/webrev.00/src/hotspot/share/memory/metaspace.cpp.udiff.html
>         <http://cr.openjdk.java.net/~zgu/8191931/webrev.00/src/hotspot/share/memory/metaspace.cpp.udiff.html>
> 
>         Not sure, but would this test not break if the class loader
>         allocates a metablock which does not fit into a specialized
>         chunk before reaching
>         _anon_and_delegating_metadata_specialize_chunk_limit - which is
>         a theoretically valid scenario?
> 
>         See calc_chunk_size():
> 
>             if ((_space_type == Metaspace::AnonymousMetaspaceType ||
>         _space_type == Metaspace::ReflectionMetaspaceType) &&
>                 _mdtype == Metaspace::NonClassType &&
>                 sum_count_in_chunks_in_use(SpecializedIndex) <
>         _anon_and_delegating_metadata_specialize_chunk_limit &&
>                 word_size + Metachunk::overhead() <= SpecializedChunk)
>         {             <<<<
>               return SpecializedChunk;
>             }
> 
>     Oops, I completely messed up here.
> 
>     I guess there is *not* reliable way to test this, cause it can jump
>     to any size in theory ...
> 
>     Thanks,
> 
>     -Zhengyu
> 
> 
> 
>         In other words, encountering small/medium chunks before reaching
>         the spec chunk limit could be, at least according to
>         calc_chunk_size(), a valid albeit rare scenario.
> 
>         --
> 
>         nits:
> 
>         "should use up to"  -> "should use at least"
>         "midium" -> "medium"
> 
>         Should the comment not be:
> 
>         -// _anon_and_delegating_metadata_specialize_chunk_limit small
>         chunks before jump to midium chunks,
>         +// _anon_and_delegating_metadata_specialize_chunk_limit
>         specialized chunks before jump to medium chunks,
> 
>         ?
> 
>         Do we always jump right to medium chunks from specialized chunks?
> 
>         --
> 
>         If your intention is to test "there shall be no small/medium
>         chunks before we have reached the limit of specialized chunks",
>         would it not be easier to test just that:
> 
>         if (sum_count_in_chunks_in_use(MediumIndex) > 0) {
>             assert(sum_count_in_chunks_in_use(SpecializedIndex) >= limit)
>         }
> 
>         instead of calculating and comparing the capacity?
> 
>         Best Regards, Thomas
> 
> 
> 
>         On Thu, Mar 15, 2018 at 3:52 PM, Zhengyu Gu <zgu at redhat.com
>         <mailto:zgu at redhat.com> <mailto:zgu at redhat.com
>         <mailto:zgu at redhat.com>>> wrote:
> 
>              Hi,
> 
>              Please review a new gtest to ensure correct sizing for
>         anonymous and
>              reflection delegating classloade's metadata space.
> 
>              Bug: https://bugs.openjdk.java.net/browse/JDK-8191931
>         <https://bugs.openjdk.java.net/browse/JDK-8191931>
>              <https://bugs.openjdk.java.net/browse/JDK-8191931
>         <https://bugs.openjdk.java.net/browse/JDK-8191931>>
>              Webrev:
>         http://cr.openjdk.java.net/~zgu/8191931/webrev.00/index.html
>         <http://cr.openjdk.java.net/~zgu/8191931/webrev.00/index.html>
>             
>         <http://cr.openjdk.java.net/~zgu/8191931/webrev.00/index.html
>         <http://cr.openjdk.java.net/~zgu/8191931/webrev.00/index.html>>
> 
> 
>              Thanks,
> 
>              -Zhengyu
> 
> 
>