[9] RFR(M): 8029799: vm/mlvm/anonloader/stress/oome prints warning: CodeHeap: # of free blocks > 10000

[Christian Thalinger]

On Feb 5, 2014, at 10:57 AM, Vladimir Kozlov <vladimir.kozlov at oracle.com> wrote:

> On 2/5/14 8:28 AM, Albert wrote:
>> Hi Vladimir,
>> 
>> thanks for looking at this. I've done the proposed measurements. The
>> code which I used to
>> get the data is included in the following webrev:
>> 
>> http://cr.openjdk.java.net/~anoll/8029799/webrev.01/
> 
> Good.
> 
>> 
>> I think some people might be interested in getting that data, so we
>> might want to keep
>> that additional output. The exact output format can be changed later
>> (JDK-8005885).
> 
> I agree that it is useful information.
> 
>> 
>> Here are the results:
>> 
>> - failing test case:
>>    - original: allocated in freelist: 2168kB, unused bytes in CodeBlob:
>> 818kB,   max_used: 21983kB
>>    - patch   : alloacted in freelist: 1123kB, unused bytes in CodeBlob:
>> 2188kB, max_used: 17572kB
>> - nashorn:
>>   - original : allocated in freelist: 2426kB, unused bytes in CodeBlob:
>> 1769kB, max_used: 201886kB
>>   - patch    : allocated in freelist: 1150kB, unused bytes in CodeBlob:
>> 3458kB, max_used: 202394kB
>> - SPECJVM2008: compiler.compiler:
>>   - original  : allocated in freelist:  168kB, unused bytes in
>> CodeBlob: 342kB, max_used: 19837kB
>>   - patch     : allocated in freelist:  873kB, unused bytes in
>> CodeBlob: 671kB, max_used: 21184kB
>> 
>> The minimum size that can be allocated from the code cache is
>> platform-dependent.
>> I.e., the minimum size depends on CodeCacheSegmentSize and
>> CodeCacheMinBlockLength.
>> On x86, for example, the min. allocatable size from the code cache is
>> 64*4=256bytes.
> 
> There is this comment in CodeHeap::search_freelist():
>  // Don't leave anything on the freelist smaller than CodeCacheMinBlockLength.
> 
> What happens if we scale down CodeCacheMinBlockLength when we increase CodeCacheSegmentSize to keep the same bytes size of minimum block?:
> 
> +     FLAG_SET_DEFAULT(CodeCacheSegmentSize, CodeCacheSegmentSize * 2);
> +     FLAG_SET_DEFAULT(CodeCacheMinBlockLength, CodeCacheMinBlockLength/2);
> 
> Based on your table below those small nmethods will use only 256 bytes blocks instead of 512 (128*4).
> 
> Note for C1 in Client VM CodeCacheMinBlockLength is 1. I don't know why for C2 it is 4. Could you also try CodeCacheMinBlockLength = 1?
> 
> All above is with CodeCacheSegmentSize 128 bytes.
> 
>> The size of adapters ranges from 400b to 600b.
>> Here is the beginning of the nmethod size distribution of the failing
>> test case:
>> 
> 
> Is it possible it is in segments number and not in bytes? If it really bytes what such (32-48 bytes) nmethods look like?

This is just a guess but these methods could be method handle trampolines.  They are very small.

> 
> Thanks,
> Vladimir
> 
>> 
>> nmethod size distribution (non-zombie java)
>> -------------------------------------------------
>> 0-16 bytes                                0[bytes]
>> 16-32 bytes                                0
>> 32-48 bytes                                45
>> 48-64 bytes                                0
>> 64-80 bytes                                41
>> 80-96 bytes                                0
>> 96-112 bytes                               6247
>> 112-128 bytes                               0
>> 128-144 bytes                               249
>> 144-160 bytes                               0
>> 160-176 bytes                               139
>> 176-192 bytes                               0
>> 192-208 bytes                               177
>> 208-224 bytes                               0
>> 224-240 bytes                               180
>> 240-256 bytes                               0
>> ...
>> 
>> 
>> I do not see a problem for increasing the CodeCacheSegmentSize if tiered
>> compilation
>> is enabled.
>> 
>> What do you think?
>> 
>> 
>> Best,
>> Albert
>> 
>> 
>> On 02/04/2014 05:52 PM, Vladimir Kozlov wrote:
>>> I think the suggestion is reasonable since we increase CodeCache *5
>>> for Tiered.
>>> Albert, is it possible to collect data how much space is wasted in %
>>> before and after this change: free space in which we can't allocate +
>>> unused bytes at the end of nmethods/adapters? Can we squeeze an
>>> adapter into 64 bytes?
>>> 
>>> Thanks,
>>> Vladimir
>>> 
>>> On 2/4/14 7:41 AM, Albert wrote:
>>>> Hi,
>>>> 
>>>> could I get reviews for this patch (nightly failure)?
>>>> 
>>>> webrev: http://cr.openjdk.java.net/~anoll/8029799/webrev.00/
>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8029799
>>>> 
>>>> problem: The freelist of the code cache exceeds 10'000 items, which
>>>> results in a VM warning.
>>>>                 The problem behind the warning is that the freelist
>>>> is populated by a large number
>>>>                 of small free blocks. For example, in failing test
>>>> case (see header), the freelist grows
>>>>                 up to more than 3500 items where the largest item on
>>>> the list is 9 segments (one segment
>>>>                 is 64 bytes). That experiment was done on my laptop.
>>>> Such a large freelist can indeed be
>>>>                 a performance problem, since we use a linear search
>>>> to traverse the freelist.
>>>> solution:  One way to solve the problem is to increase the minimal
>>>> allocation size in the code cache.
>>>>                 This can be done by two means: we can increase
>>>> CodeCacheMinBlockLength and/or
>>>>                 CodeCacheSegmentSize. This patch follows the latter
>>>> approach, since increasing
>>>>                 CodeCacheSegmentSize decreases the size that is
>>>> required by the segment map. More
>>>>                 concretely, the patch doubles the
>>>> CodeCacheSegmentSize from 64 byte to 128 bytes
>>>>                 if tiered compilation is enabled.
>>>>                 The patch also contains an optimization in the
>>>> freelist search (stop searching if we found
>>>>                 the appropriate size) and contains some code cleanups.
>>>> testing:    With the proposed change, the size of the freelist is
>>>> reduced to 200 items. There is only
>>>>                 a slight increase in memory required by code cache
>>>> by at most 3% (all data measured
>>>>                 for the failing test case on a Linux 64-bit system,
>>>> 4 cores).
>>>>                 To summarize, increasing the minimum allocation size
>>>> in the code cache results in
>>>>                 potentially more unused memory in the code cache due
>>>> to unused bits at the end of
>>>>                 an nmethod. The advantage is that we potentially
>>>> have less fragmentation.
>>>> 
>>>> proposal: - I think we could remove CodeCacheMinBlockLength without
>>>> loss of generality or usability
>>>>                   and instead adapt the parameter
>>>> CodeCacheSegmentSize at Vm startup.
>>>>                   Any opinions?
>>>> 
>>>> Many thanks in advance,
>>>> Albert