[9] RFR(M): 8029799: vm/mlvm/anonloader/stress/oome prints warning: CodeHeap: # of free blocks > 10000

[Vladimir Kozlov]

On 2/5/14 8:28 AM, Albert wrote:
> Hi Vladimir,
>
> thanks for looking at this. I've done the proposed measurements. The
> code which I used to
> get the data is included in the following webrev:
>
> http://cr.openjdk.java.net/~anoll/8029799/webrev.01/

Good.

>
> I think some people might be interested in getting that data, so we
> might want to keep
> that additional output. The exact output format can be changed later
> (JDK-8005885).

I agree that it is useful information.

>
> Here are the results:
>
> - failing test case:
>     - original: allocated in freelist: 2168kB, unused bytes in CodeBlob:
> 818kB,   max_used: 21983kB
>     - patch   : alloacted in freelist: 1123kB, unused bytes in CodeBlob:
> 2188kB, max_used: 17572kB
> - nashorn:
>    - original : allocated in freelist: 2426kB, unused bytes in CodeBlob:
> 1769kB, max_used: 201886kB
>    - patch    : allocated in freelist: 1150kB, unused bytes in CodeBlob:
> 3458kB, max_used: 202394kB
> - SPECJVM2008: compiler.compiler:
>    - original  : allocated in freelist:  168kB, unused bytes in
> CodeBlob: 342kB, max_used: 19837kB
>    - patch     : allocated in freelist:  873kB, unused bytes in
> CodeBlob: 671kB, max_used: 21184kB
>
> The minimum size that can be allocated from the code cache is
> platform-dependent.
> I.e., the minimum size depends on CodeCacheSegmentSize and
> CodeCacheMinBlockLength.
> On x86, for example, the min. allocatable size from the code cache is
> 64*4=256bytes.

There is this comment in CodeHeap::search_freelist():
   // Don't leave anything on the freelist smaller than 
CodeCacheMinBlockLength.

What happens if we scale down CodeCacheMinBlockLength when we increase 
CodeCacheSegmentSize to keep the same bytes size of minimum block?:

+     FLAG_SET_DEFAULT(CodeCacheSegmentSize, CodeCacheSegmentSize * 2);
+     FLAG_SET_DEFAULT(CodeCacheMinBlockLength, CodeCacheMinBlockLength/2);

Based on your table below those small nmethods will use only 256 bytes 
blocks instead of 512 (128*4).

Note for C1 in Client VM CodeCacheMinBlockLength is 1. I don't know why 
for C2 it is 4. Could you also try CodeCacheMinBlockLength = 1?

All above is with CodeCacheSegmentSize 128 bytes.

> The size of adapters ranges from 400b to 600b.
> Here is the beginning of the nmethod size distribution of the failing
> test case:
>

Is it possible it is in segments number and not in bytes? If it really 
bytes what such (32-48 bytes) nmethods look like?

Thanks,
Vladimir

>
> nmethod size distribution (non-zombie java)
> -------------------------------------------------
> 0-16 bytes                                0[bytes]
> 16-32 bytes                                0
> 32-48 bytes                                45
> 48-64 bytes                                0
> 64-80 bytes                                41
> 80-96 bytes                                0
> 96-112 bytes                               6247
> 112-128 bytes                               0
> 128-144 bytes                               249
> 144-160 bytes                               0
> 160-176 bytes                               139
> 176-192 bytes                               0
> 192-208 bytes                               177
> 208-224 bytes                               0
> 224-240 bytes                               180
> 240-256 bytes                               0
> ...
>
>
> I do not see a problem for increasing the CodeCacheSegmentSize if tiered
> compilation
> is enabled.
>
> What do you think?
>
>
> Best,
> Albert
>
>
> On 02/04/2014 05:52 PM, Vladimir Kozlov wrote:
>> I think the suggestion is reasonable since we increase CodeCache *5
>> for Tiered.
>> Albert, is it possible to collect data how much space is wasted in %
>> before and after this change: free space in which we can't allocate +
>> unused bytes at the end of nmethods/adapters? Can we squeeze an
>> adapter into 64 bytes?
>>
>> Thanks,
>> Vladimir
>>
>> On 2/4/14 7:41 AM, Albert wrote:
>>> Hi,
>>>
>>> could I get reviews for this patch (nightly failure)?
>>>
>>> webrev: http://cr.openjdk.java.net/~anoll/8029799/webrev.00/
>>> bug: https://bugs.openjdk.java.net/browse/JDK-8029799
>>>
>>> problem: The freelist of the code cache exceeds 10'000 items, which
>>> results in a VM warning.
>>>                  The problem behind the warning is that the freelist
>>> is populated by a large number
>>>                  of small free blocks. For example, in failing test
>>> case (see header), the freelist grows
>>>                  up to more than 3500 items where the largest item on
>>> the list is 9 segments (one segment
>>>                  is 64 bytes). That experiment was done on my laptop.
>>> Such a large freelist can indeed be
>>>                  a performance problem, since we use a linear search
>>> to traverse the freelist.
>>> solution:  One way to solve the problem is to increase the minimal
>>> allocation size in the code cache.
>>>                  This can be done by two means: we can increase
>>> CodeCacheMinBlockLength and/or
>>>                  CodeCacheSegmentSize. This patch follows the latter
>>> approach, since increasing
>>>                  CodeCacheSegmentSize decreases the size that is
>>> required by the segment map. More
>>>                  concretely, the patch doubles the
>>> CodeCacheSegmentSize from 64 byte to 128 bytes
>>>                  if tiered compilation is enabled.
>>>                  The patch also contains an optimization in the
>>> freelist search (stop searching if we found
>>>                  the appropriate size) and contains some code cleanups.
>>> testing:    With the proposed change, the size of the freelist is
>>> reduced to 200 items. There is only
>>>                  a slight increase in memory required by code cache
>>> by at most 3% (all data measured
>>>                  for the failing test case on a Linux 64-bit system,
>>> 4 cores).
>>>                  To summarize, increasing the minimum allocation size
>>> in the code cache results in
>>>                  potentially more unused memory in the code cache due
>>> to unused bits at the end of
>>>                  an nmethod. The advantage is that we potentially
>>> have less fragmentation.
>>>
>>> proposal: - I think we could remove CodeCacheMinBlockLength without
>>> loss of generality or usability
>>>                    and instead adapt the parameter
>>> CodeCacheSegmentSize at Vm startup.
>>>                    Any opinions?
>>>
>>> Many thanks in advance,
>>> Albert
>>>
>