Possibility of code cache unloading

[Eric Caspole]

Tom,
Thanks for your comments. Your idea is a good extension of what I  
have been playing with. I am going to do some experiments on the big  
apps I have access to.
Eric


On Oct 12, 2009, at 12:45 PM, Tom Rodriguez wrote:

> There are two main problems.  First is how to identify nmethods  
> which aren't being used without penalizing execution too much and  
> also without introducing new races.  Reclamation of nmethods is  
> somewhat tricky and reusing existing mechanisms would be good.   
> Second is the policy part.  Reclamation of nmethods isn't prompt so  
> you have to get enough warning that the code cache is getting full  
> to start reclamation before you run out.  It's basically a GC  
> policy problem though we can always simply delay compilation until  
> some storage has been reclaimed.  A hiccup near the transition  
> would still be better than our current policy of simply stopping  
> compilation.
>
> For the first part, I'd always imagined we end up with some  
> strategy where we marked methods as speculatively not entrant and  
> if they weren't used within some time period we'd transition them  
> fully to not entrant and reclaim the nmethod storage.  One problem  
> with this is the repatching of the nmethod back into a normal  
> state.  I guess if it's done at a safepoint it wouldn't be a big deal.
>
> On the other hand it occurred to me today that we could rely on the  
> same guarantees that the nmethod sweeper uses to do this without  
> requiring patching.  There are only two ways to get into compiled  
> code.  You either grab the nmethod entry point from the methodOop  
> you are invoking, which is how interpreter invokes and polymorphic  
> dispatch work, or you have a CompiledIC that jumps you there  
> directly which is the normal way compiled code works.  If you break  
> both of these kinds of links then an nmethod can no longer be  
> invoked.  This is essentially how making an nmethod not entrant  
> works but since the cleaning of inline caches is done at a  
> safepoint we also patch the entry point of the nmethod to dispatch  
> back into the call resolution code.  This allows the cleaning of  
> the CompiledICs to be done lazily because any caller that still  
> references the nmethod will still end up back in the resolution code.
>
> I think we could use the same strategy for identifying unused  
> nmethods.  We simply break the reference from the methodOop to the  
> nmethod and start a cache cleaning but without patching the entry  
> point.  If we ever invoke those nmethods again we'll end up going  
> through the call resolution code where we can detect that the  
> nmethod is really being used and simply restore the nmethod and  
> continue using it.  If it's never referenced within some time  
> period the nmethod can simply be freed because we know that it's  
> clean.
>
> As far as policy goes we could simply mark every nmethod this way  
> when we get into trouble but I think we'd want a more GC like  
> policy that would begin identifying nmethods once the code cache  
> started filling up and reclaim them before we actually run out.
>
> tom
>
> On Oct 12, 2009, at 9:53 AM, Eric Caspole wrote:
>
>> We discovered some app servers do not have really well behaved  
>> class loader schemes, so that in a long running process with  
>> multiple re-deployments of the same web app, the old stuff never  
>> gets unloaded and the first thing to run out of space in this case  
>> was the code cache. In this event the compiler shuts itself off  
>> and does not turn itself on regardless of what happens in later GC/ 
>> sweep cycles.
>>
>> We have been wondering what it would take to do some kind of code  
>> cache unloading so long running applications like this won't end  
>> up having the later redeployments running interpreter-only. Many  
>> users do not even know when this has happened and may or may not  
>> notice a gradual mysterious slowdown until they just restart the  
>> process.
>>
>> In various discussions ideas have popped up from marking some  
>> amount of existing nmethods non-entrant so they will get unloaded  
>> in the existing code cache, to more elaborate reallocation schemes  
>> for the whole code cache. If necessary, a short term slowdown  
>> before getting back to having all the hot methods recompiled seems  
>> better than restarting the process. Perhaps there could be some  
>> JMX notification for this situation.
>>
>> One idea that came out of our testing was the idea of a current  
>> working set of hot methods. We saw that if enough space could be  
>> regained in the code cache, the program's normal operation would  
>> require recompiles only of the current hot method set, which is  
>> hopefully a lot smaller than the whole code cache size. Then it  
>> would quickly resume normal operation, and only in the event of  
>> one or more web app redeployments as mentioned above would the  
>> code cache require another flush operation, hopefully days or  
>> weeks later.
>>
>> Lastly, it is probably desirable to have a fallback plan of giving  
>> up and shutting off the compiler if the flush cycles are happening  
>> too often, for example if the hot method working set size is too  
>> close to the whole code cache size, and the application  
>> performance won't be any worse than it gets today.
>>
>> We'd like to hear if anyone else has a strong opinion or great  
>> idea on this topic, and what corner case did we not think of. I  
>> remember a wise crack about this topic at the JVM Languages Summit  
>> a few weeks ago so it seems someone out there is thinking about it.
>>
>> Thanks,
>> Eric
>>
>>
>
>