RFR (M): 8006952: Slow VM due to excessive code cache freelist iteration

[Nils Eliasson]

Vladimir and Andreas, thanks for your feedback! I am considering your 
suggestions.

My fix is a step in the right direction but I realize it is not enough.

Initially, it works good reserving the last part of the heap block. But 
if the application has a bigger active code footprint that what can fit 
in the code cache, there will be new flushes that will increase 
fragmentation and slowly reduce the usable part of the code cache. So I 
will add functionality for backing off.

//Nils

Schoesser, Andreas skrev 2013-02-01 10:58:
> Hi Nils,
>
> this is IMO a good idea since we experienced the same problem with one of our ports. I think in our case it was not the nmethod sweeper continuously running (there's MinCodeCacheFlushingInterval to only speculatively disconnect nmethods every 30 seconds) but from the compiler loop constantly polling the freelist without the nmethod sweeper making any progress.
>
> You also might consider to propagate the "critical" flag down to search_freelist(). Then, a simple address compare would do to keep the end of the code cache free from nmethods:
> - For non-critical blobs, search_freelist() may only fit free blocks < CodeCache::end() - CodeCacheMinimumFreeSpace
> - For critical blocks, search_freelist() may fit all free blocks
>
> Best regards,
> Andreas
>
> -----Original Message-----
> From: hotspot-compiler-dev-bounces at openjdk.java.net [mailto:hotspot-compiler-dev-bounces at openjdk.java.net] On Behalf Of Nils Eliasson
> Sent: Donnerstag, 31. Januar 2013 19:41
> To: hotspot-compiler-dev at openjdk.java.net
> Subject: Re: RFR (M): 8006952: Slow VM due to excessive code cache freelist iteration
>
> http://cr.openjdk.java.net/~neliasso/8006952/webrev.04/
>
> //N
>
> Vladimir Kozlov skrev 2013-01-30 21:16:
>> Nils, you sent webrev for other bug fix.
>>
>> Vladimir
>>
>> On 1/30/13 7:43 AM, Nils Eliasson wrote:
>>> Thanks for the link, interesting background.
>>>
>>> Even if finding the largest block on the freelist is free, we would
>>> still have a misbehavior when the free heap block is less than 1,5M and
>>> the flushing begins to create a 1,5M freelist block.
>>>
>>> Unallocated memory is a good measurement if have some continuous memory
>>> in reserve for critical blobs. I am thinking of using the thresholds
>>> against unallocated memory, but forbidding nmethods from using the last
>>> 0,5M (MinimumFreeSpace) from the heap block. That would prevent the VM
>>> from fragmenting the last continuous memory space, and allow the VM to
>>> use all of the freelist.
>>>
>>> Like this:
>>> http://cr.openjdk.java.net/~neliasso/8002364/webrev.03
>>>
>>> //Nils
>>>
>>> Vladimir Kozlov skrev 2013-01-29 20:35:
>>>> Nils,
>>>>
>>>> You are reversing 7025742 fix. I would prefer to keep track of a
>>>> largest free block during allocation in CodeCache (by using ordered
>>>> list or other technique) to avoid scanning the list.
>>>>
>>>> Vladimir
>>>>
>>>> On 1/29/13 2:56 AM, Nils Eliasson wrote:
>>>>> Remove continuous free block requirement for code cache flushing and
>>>>> minimum free space.
>>>>>
>>>>> This causes a degenerate behavior when the VM are repeatedly flushing
>>>>> the code cache trying to free up a continuous 1,5M block. Since the
>>>>> freelist is fragmented a significant part of the code cache is kept
>>>>> unallocated. A significant amount of time is spent traversing the
>>>>> freelist holding the code cache lock or waiting for other threads
>>>>> doing
>>>>> the same.
>>>>>
>>>>> I don't see any reason for why a continuous block is required. Code
>>>>> cache flushing threshold (CodeCacheFlushingMinimumFreeSpace) is
>>>>> default
>>>>> 1,5M and code cache minimum threshold (CodeCacheMinimumFreeSpace) is
>>>>> 0,5M. Finding such a block on the freelist is often unrealistic,
>>>>> and has
>>>>> not any purpose. The largest nmethods are in the order of 250k, and
>>>>> they
>>>>> are not allocated when code cache is starting to run out. All critical
>>>>> adapters are small and will fit easily in the freelist or in the
>>>>> remaining heap space.
>>>>>
>>>>> http://cr.openjdk.java.net/~neliasso/8006952/webrev.02/
>>>>>
>>>>> Thanks,
>>>>> Nils Eliasson