RFR (L): 8015774: Add support for multiple code heaps

[Christian Thalinger]

Some general comments:

Don't use T1, T2, … to name the tiers because SPARC processors are named like this.  It's confusing.  Use tier 1, tier 2, … instead.  The compilation policies actually talk about tier levels...

I was hoping that we can have a more loose definition of the code heaps.  So we can easily add more.  What this patch does is a good start but all the hardcoded values and fields need to go at a later point in time, in my opinion.

src/share/vm/code/codeBlob.hpp:
+// CodeBlob Types
+// Used in the CodeCache to assign CodeBlobs to different CodeHeaps
+enum CodeBlobType {
+  btMethodNoProfile   = 0,    // T1 and T4 methods (including native methods)
+  btMethodProfile     = 1,    // T2 and T3 methods with profile information
+  btNonMethod         = 2     // Non-methods like Buffers, Adapters and Runtime Stubs
+};
The bt prefix is already overloaded (e.g. BasicType).  Generally speaking, I never liked the fact that enums don't have their name as scope.  Maybe we can agree on doing something like this:

struct CodeBlobType {
  enum Enum {
    NonMethod = 2
  };
};

void foo() {
  int a = CodeBlobType::NonMethod;
  int b = (CodeBlobType::Enum) 1;
}

Or we move to C++11 :-)

http://www.stroustrup.com/C++11FAQ.html#enum

src/share/vm/code/codeCache.cpp:
+#define FOR_ALL_HEAPS(index) for (int index = 0; index < _heaps->length(); ++index)
I'm one of the people who started to hate macros (for various reasons).  Although I see the point in defining these I'd rather go without them.  Is there a way we can use STL vectors with our allocators?  Or we could write our own iterator class which iterates either over GrowableArrays in general or the code heaps in particular.

src/share/vm/memory/heap.hpp:
+  const char* getName() const   { return _name; }
This should either be get_name() or name().

Let's start the discussion and I'll do another review round later.

On Oct 2, 2013, at 7:06 AM, Tobias Hartmann <tobi.hartmann at gmail.com> wrote:

> Hi,
> 
> please review the following change. 
> 
> bug: https://bugs.openjdk.java.net/browse/JDK-8015774
> webrev: http://cr.openjdk.java.net/~anoll/8015774/webrev.00/
> 
> This change implements support for multiple code heaps in the code cache. The interface of the code cache was changed accordingly and references from other components of the VM were adapted. This includes the indirect references from:
> - the Serviceability Agent: vmStructs and the Java code cache interface (sun.jvm.hotspot.code.CodeCache)
> - the dtrace ustack helper script (jhelper.d)
> - the pstack support library libjvm_db.c
> 
> Currently the code cache contains the following three code heaps each of which contains CodeBlobs of a specific type:
> - Non-Profiled methods: nmethods that are not profiled, i.e., those compiled at tier 1 or 4 and native methods
> - Profiled methods: nmethods that are profiled, i.e., those compiled at tier 2 or 3
> - Non-methods: Non-methods like Buffers, Adapters and Runtime Stubs
> 
> By default, 2/3 of the ReservedCodeCacheSize is used for the non-profiled heap and 1/3 is used for the profiled heap. Experiments with a small code cache size have shown that this configuration performs best (see [1]). Sizes can be configured using the NonProfiledCodeHeapSize and ProfiledCodeHeapSize parameters. By now the non-method heap has a fixed size of 8mb. More tests have to be performed to determine reasonable default values for different build and runtime configurations. It would be also possible to add an additional heap for the CodeBlobs needed during VM startup (adapters, stubs, interpreter, ...) and use the non-method heap for non-method blobs allocated during runtime.
> 
> The main benefit of this allocation is that functions operating on nmethods can now be optimized to only iterate over CodeBlobs on the nmethod heaps, avoiding a full scan of the code cache including the non-methods. Performance evaluation shows that this results in a great reduction of the sweeping time. Further it is now possible to extend the sweeper to selectively sweep nmethods of different compilation levels. Because it is more expensive to recompile a highly optimized method, it makes sense to first sweep those compiled at a lower level. Possible future optimizations may also include multithreaded sweeping and separation of nmethod code and metadata.
> 
> The code was tested using Nashorn + Octane, DaCapo, SPECJvm2008 and jprt.
> 
> Benchmark results:
> - [2] shows the score of the old and new version running Nashorn with the Octane benchmark at different code cache sizes (10 runs each, sizes < 32mb crash with the old version). An performance improvement of around 15% is reached.
> - [3] shows the sweep time of the NMethodSweeper during runs of the Octane benchmark with different code cache sizes (10 runs each). The time is greatly reduced with the new version. 
> - [4] shows the time ratio (TOld / TNew - 1) of the DaCapo benchmarks with a code cache size of 64mb (30 runs with 20 warmups each). With the time differences being smaller than the confidence intervals it is not possible determine a difference between the two versions.
> - Multiple runs of the SPECJvm2008 benchmark with different code cache sizes did not show a difference between the two versions. This is probably due to the fact that not much code is compiled at runtime compared to Nashorn with the Octane benchmark.
> 
> The error bars at each data point show the 95% confidence interval.
> 
> [1] https://bugs.openjdk.java.net/secure/attachment/16372/OctaneRatio.png
> [2] https://bugs.openjdk.java.net/secure/attachment/16374/OctaneCCSizes_new.png
> [3] https://bugs.openjdk.java.net/secure/attachment/16373/OctaneSweepTime_new.png
> [4] https://bugs.openjdk.java.net/secure/attachment/16371/DaCapoCCSizes_64.png
> 
> Thanks and best regards,
> 
> Tobias

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/hotspot-compiler-dev/attachments/20131003/a2e16be4/attachment.html