RFR (L): 8015774: Add support for multiple code heaps

[Vladimir Kozlov]

ProfiledCodeHeapSize value is missleading because part of it will be cut 
for non_method section:

profile_size = ProfiledCodeHeapSize - non_method_size;

And you hided default non_method_size (8Mb) inside the code:

+  size_t non_method_size = 
ReservedCodeSpace::allocation_align_size_up(MIN2(ReservedCodeCacheSize, 
8*M));

It is not good. If in future we add more stubs, for example, 8Mb could 
be not enough and it will be hard to find this place. May be

NonProfiledCodeHeapSize + ProfiledCodeHeapSize == ReservedCodeCacheSize 
+ 8Mb

So the size could be adjusted by setting 2 flags.

Why you need all heaps for Client VM (only C1 is available)?

+bool CodeCache::heap_available(CodeBlobType bt) {
+  if (TieredCompilation || bt == btNonMethod) {
+    // Use all heaps for TieredCompilation
+    return true;
+  } else {
+#ifdef COMPILER2
+    // We only need the heap for C2
+    return (bt == btMethodNoProfile);
+#else
+    // We need both heaps
+    return true;
+#endif
+  }
  }

report_codemem_full(). When I discussed this feature with Igor, he told 
me that if you failed allocate in one heap you can allocate in an other 
heap. Or you can move boundary. The longer you run "stable" application 
the less ProfiledCodeHeap will be used (could be changed later when we 
add code to fall back to profiled code during nmethod deoptimization). 
It would be nice to redirect its space for NonProfiledCodeHeap. For this 
purpose layout of heaps should be reversed.
It would be strange to see free space in whole codecache but not able to 
allocate.

Next message is incorrect now since you need heap's specific flag:

+    warning("Try increasing the code cache size using 
-XX:ReservedCodeCacheSize=");

thanks,
Vladimir

On 10/2/13 7:06 AM, Tobias Hartmann wrote:
> Hi,
>
> please review the following change.
>
> bug: https://bugs.openjdk.java.net/browse/JDK-8015774
> webrev: http://cr.openjdk.java.net/~anoll/8015774/webrev.00/
>
> This change implements support for multiple code heaps in the code
> cache. The interface of the code cache was changed accordingly and
> references from other components of the VM were adapted. This includes
> the indirect references from:
> - the Serviceability Agent: vmStructs and the Java code cache interface
> (sun.jvm.hotspot.code.CodeCache)
> - the dtrace ustack helper script (jhelper.d)
> - the pstack support library libjvm_db.c
>
> Currently the code cache contains the following three code heaps each of
> which contains CodeBlobs of a specific type:
> - Non-Profiled methods: nmethods that are not profiled, i.e., those
> compiled at tier 1 or 4 and native methods
> - Profiled methods: nmethods that are profiled, i.e., those compiled at
> tier 2 or 3
> - Non-methods: Non-methods like Buffers, Adapters and Runtime Stubs
>
> By default, 2/3 of the ReservedCodeCacheSize is used for the
> non-profiled heap and 1/3 is used for the profiled heap. Experiments
> with a small code cache size have shown that this configuration performs
> best (see [1]). Sizes can be configured using the
> NonProfiledCodeHeapSize and ProfiledCodeHeapSize parameters. By now the
> non-method heap has a fixed size of 8mb. More tests have to be performed
> to determine reasonable default values for different build and runtime
> configurations. It would be also possible to add an additional heap for
> the CodeBlobs needed during VM startup (adapters, stubs, interpreter,
> ...) and use the non-method heap for non-method blobs allocated during
> runtime.
>
> The main benefit of this allocation is that functions operating on
> nmethods can now be optimized to only iterate over CodeBlobs on the
> nmethod heaps, avoiding a full scan of the code cache including the
> non-methods. Performance evaluation shows that this results in a great
> reduction of the sweeping time. Further it is now possible to extend the
> sweeper to selectively sweep nmethods of different compilation levels.
> Because it is more expensive to recompile a highly optimized method, it
> makes sense to first sweep those compiled at a lower level. Possible
> future optimizations may also include multithreaded sweeping and
> separation of nmethod code and metadata.
>
> The code was tested using Nashorn + Octane, DaCapo, SPECJvm2008 and jprt.
>
> Benchmark results:
> - [2] shows the score of the old and new version running Nashorn with
> the Octane benchmark at different code cache sizes (10 runs each, sizes
> < 32mb crash with the old version). An performance improvement of around
> 15% is reached.
> - [3] shows the sweep time of the NMethodSweeper during runs of the
> Octane benchmark with different code cache sizes (10 runs each). The
> time is greatly reduced with the new version.
> - [4] shows the time ratio (TOld / TNew - 1) of the DaCapo benchmarks
> with a code cache size of 64mb (30 runs with 20 warmups each). With the
> time differences being smaller than the confidence intervals it is not
> possible determine a difference between the two versions.
> - Multiple runs of the SPECJvm2008 benchmark with different code cache
> sizes did not show a difference between the two versions. This is
> probably due to the fact that not much code is compiled at runtime
> compared to Nashorn with the Octane benchmark.
>
> The error bars at each data point show the 95% confidence interval.
>
> [1] https://bugs.openjdk.java.net/secure/attachment/16372/OctaneRatio.png
> [2]
> https://bugs.openjdk.java.net/secure/attachment/16374/OctaneCCSizes_new.png
> [3]
> https://bugs.openjdk.java.net/secure/attachment/16373/OctaneSweepTime_new.png
> [4]
> https://bugs.openjdk.java.net/secure/attachment/16371/DaCapoCCSizes_64.png
>
> Thanks and best regards,
>
> Tobias