RFR (L): 8015774: Add support for multiple code heaps

Igor Veresov igor.veresov at oracle.com
Thu Apr 17 22:10:41 UTC 2014 

Hi Tobias,

See replies inline.

On Apr 15, 2014, at 2:11 AM, Tobias Hartmann <tobias.hartmann at oracle.com> wrote:

> Hi Igor,
> 
> On 04/15/2014 12:15 AM, Igor Veresov wrote:
>> codeCache.hpp:
>> I guess CodeCache::_save_nmethods is not longer necessary since we ripped out the speculative disconnect?
> 
> Yes, seems like this reappeared during merging. I removed it.
> 
>> codeCache.cpp:
>> 
>> Why are these necessary?
>>   51 #ifdef COMPILER1
>>   52 #include "c1/c1_Compilation.hpp"
>>   53 #endif
>>   54 #ifdef COMPILER2
>>   55 #include "opto/compile.hpp"
>>   56 #endif
> 
> Because in CodeCache::initialize_heaps() the size of the C1 and C2 code buffers have to be determined (see lines 156 and 165) to set the size of the non-method code heap.
> 

Alright, it feels like it should be abstracted away somehow but I don’t see a good way of doing it that early in initialization.

>> arguments.cpp:
>> 
>> I know you’re working on the adaptive resizing solution. How are you planning to implement it?
>> Since it would be expensive to do compaction, the degree of resizing possible will be hampered by fragmentation. Perhaps there should be another allocation layer that gives preference to for certain allocation type to be allocated in specific areas, but does not preclude allocation in other areas? For example if we’re out of space for C2 methods we would be allowed to allocate in the C1 space. Is it something like that?
> 
> While working for my masters thesis, I already implemented a fully working patch. The dynamic resizing of the code heaps works by placing the profiled and the non-profiled code heaps adjacent to each other in memory. Both heaps grow towards the shared boundary (one towards the higher and one towards the lower addresses). Like this it is possible to move the boundary and therefore resize the code heaps, as long as there is enough free space around.
> One problem is that there may be a lot of free space in one code heap, but as long as there is no free space at the boundary we are not able to resize. One solution would be to modify the sweeper such that it tends to remove more methods at the boundary.
> 
> However, measurements showed that the dynamic resizing does not improve performance. This is partly due to an increased amount of sweeping. Further, a lot of code had to be changed to support downwards growing code heaps and the resizing of virtual spaces, leading to an increased complexity.

Could you please measure how much space to we waste for typical workloads without the resizing? Perhaps turn the flusher off and see what’s the minimum code cache size is required before and after the change? Obviously we can tune it for a particular workload, but is there a good average solution?

Simply wasting potentially a large amount of space is not good. Perhaps the allocation strategy should be more flexible (boundary moving without compaction will likely get stuck because of the fragmentation). For example it can work as follows:
- Each code cache has a free list from which it allocates. We populate these at startup according to the initial boundaries that you determined.
- There are n lists per heap to track allocated blobs. Lets call those “allocated lists”. One list is for local allocations, and then there is a list for each other heap we may try allocating from.
- When allocating you first try to allocate locally. If succeeded you splice the block from the local free list and put it on local allocated list.
- If there are no fitting local blocks, you consider the free lists of other heaps. If you can successfully allocate from there, you splice the block from the free list, and transfer it to the corresponding allocated list of the heap you allocated for.
- When you free a block you return it to the free list of the heap it was allocated from.
- Iterating over blobs for a particular heap is done by iterating over all its allocated lists.
- Optionally, may be the free lists should be sorted in address order to get nicer clustering (may be use skip lists for that?)

Or may be there is some other solution with less coding... But I think we should keep using the space as efficiently as before or close to that.

> 
>> I think if we want to push the current solution we have to bump the code cache size temporarily up until we have an allocation strategy that doesn’t waste space so much. Otherwise there’re might be regressions for applications that work fine with the current setup.
> 
> Yes, I agree.
> 
>> sweeper.cpp:
>> 
>> I agree with Vladimir, I’d be nice to make code cache iteration more abstract and not expose nmethod types when doing it.
> 
> Yes, I will have a look at it.
> 
>> globals.hpp:
>> 
>> We probably don’t need all of those spaces for the interpreter-only configuration:
>>  190 define_pd_global(intx, NonProfiledCodeHeapSize,      14*M);
>>  191 define_pd_global(intx, ProfiledCodeHeapSize,         15*M );
>>  192 define_pd_global(intx, NonMethodCodeHeapSize,        3*M );
> 
> Indeed. I think in this case we only need the non-method code heap. I tried to build an interpreter only VM with "make all_fastdebugcore" but apparently this is not supported anymore ("No (fastdebugcore) for x86"). Are there platforms where this is still supported? Or is it enough to decide according to the -Xint option?
> 

Hm.. no sure, I wan’t aware we don’t do core builds anymore. Could you use all the space for NonMethodCodeHeap in this case and for Xint ? Will setting all other guys to 0 work?

igor

> Thanks,
> Tobias
> 
>> igor
>>  
>> On Apr 14, 2014, at 6:05 AM, Tobias Hartmann <tobias.hartmann at oracle.com> wrote:
>> 
>>> Hi Volker,
>>> 
>>> please see comments inline.
>>> 
>>> On 04/14/2014 02:54 PM, Volker Simonis wrote:
>>>> Hi Tobias,
>>>> 
>>>> is this change intended for jdk8 or 9?
>>>> 
>>> the change is intended for JDK 9. The webrev is based on the JDK 9 repository (http://hg.openjdk.java.net/jdk9/hs-comp/hotspot) changeset 6280.
>>> 
>>>>  In any case, I saw that your
>>>> webrev is based on the hsx repository which I thought should not be
>>>> used any more.
>>>> 
>>> Why do you think that it is based on hsx? The earlier versions 00 - 03 actually were, because I already started developing the patch last year, but the newest one is based on the JDK 9 repository.
>>> 
>>> As Roland noticed, the default code heap sizes for arm/ppc were wrong (because tiered compilation is supported). The updated webrev can be found at:
>>> 
>>> http://cr.openjdk.java.net/~anoll/8015774/webrev.05/
>>> 
>>> Thanks,
>>> Tobias
>>> 
>>>> Could you please rebase your patch on jdk9 and also
>>>> update the corresponding ppc64 files in the webrev (as far as I can
>>>> see, that's only src/cpu/ppc/vm/c2_globals_ppc.hpp). I'd like to test
>>>> this on Linux/PPC64 and AIX as well.
>>>> 
>>>> Thank you and best regards,
>>>> Volker
>>>> 
>>>> 
>>>> 
>>>> On Mon, Apr 14, 2014 at 1:56 PM, Tobias Hartmann
>>>> 
>>>> <Tobias.Hartmann at oracle.com>
>>>>  wrote:
>>>> 
>>>>> Hi,
>>>>> 
>>>>> please review the new version of the following patch that adds support for
>>>>> multiple code heaps to the code cache.
>>>>> 
>>>>> Bug:
>>>>> https://bugs.openjdk.java.net/browse/JDK-8015774
>>>>> 
>>>>> Webrev:
>>>>> http://cr.openjdk.java.net/~anoll/8015774/webrev.04/
>>>>> 
>>>>> 
>>>>> Short description:
>>>>> This change implements support for multiple code heaps in the code cache.
>>>>> The interface of the code cache was changed accordingly and references from
>>>>> other components of the VM were adapted. This includes the indirect
>>>>> references from:
>>>>> - the Serviceability Agent: vmStructs and the Java code cache interface
>>>>> (sun.jvm.hotspot.code.CodeCache)
>>>>> - the dtrace ustack helper script (jhelper.d)
>>>>> - the pstack support library libjvm_db.c
>>>>> 
>>>>> Currently the code cache contains the following three code heaps each of
>>>>> which contains CodeBlobs of a specific type:
>>>>> - Non-Profiled methods: nmethods that are not profiled and native methods
>>>>> - Profiled methods: nmethods that are profiled
>>>>> - Non-methods: Non-methods like buffers and adapters
>>>>> 
>>>>> By default the non-method code heap uses 3 MB plus additional space for the
>>>>> compiler buffers that is dependent on the number of compiler threads (see
>>>>> CodeCache::initialize_heaps). The remaining code cache space is distributed
>>>>> equally among the non-profiled and the profiled code heaps.
>>>>> 
>>>>> Tested:
>>>>> JPRT, SPECjvm2008, SPECjbb2005, SPECjbb2013, Octane + Nashorn
>>>>> 
>>>>> Thanks,
>>>>> Tobias
>>>>> 
>>>>> 
>>>>> -------- Original Message --------
>>>>> Subject: Re: RFR (L): 8015774: Add support for multiple code heaps
>>>>> Date: Mon, 21 Oct 2013 17:47:48 +0200
>>>>> From: Albert Noll
>>>>> <albert.noll at oracle.com>
>>>>> 
>>>>> To: Azeem Jiva
>>>>> <azeem.jiva at oracle.com>
>>>>> 
>>>>> 
>>>>> 
>>>>> That is fine. As we realized the last two weeks, there is more work to be
>>>>> done to make multiple code heaps work effectively.
>>>>> 
>>>>> Albert
>>>>> 
>>>>> Von meinem iPhone gesendet
>>>>> 
>>>>> Am 21.10.2013 um 17:36 schrieb Azeem Jiva
>>>>> <azeem.jiva at oracle.com>
>>>>> :
>>>>> 
>>>>> I still think we should hold off for JDK8u20
>>>>> 
>>>>> --
>>>>> Azeem Jiva
>>>>> @javawithjiva
>>>>> 
>>>>> On Oct 21, 2013, at 8:33 AM, Albert Noll
>>>>> <albert.noll at oracle.com>
>>>>>  wrote:
>>>>> 
>>>>> 
>>>>> Von meinem iPhone gesendet
>>>>> 
>>>>> Anfang der weitergeleiteten E‑Mail:
>>>>> 
>>>>> Von: Vladimir Kozlov
>>>>> <vladimir.kozlov at oracle.com>
>>>>> 
>>>>> Datum: 11. Oktober 2013 19:38:07 MESZ
>>>>> An:
>>>>> hotspot-compiler-dev at openjdk.java.net
>>>>> 
>>>>> Betreff: Re: RFR (L): 8015774: Add support for multiple code heaps
>>>>> 
>>>>> This looks acceptable.
>>>>> 
>>>>> Thanks,
>>>>> Vladimir
>>>>> 
>

More information about the hotspot-compiler-dev mailing list