Possibility of code cache unloading

[Tom Rodriguez]

There are two main problems.  First is how to identify nmethods which  
aren't being used without penalizing execution too much and also  
without introducing new races.  Reclamation of nmethods is somewhat  
tricky and reusing existing mechanisms would be good.  Second is the  
policy part.  Reclamation of nmethods isn't prompt so you have to get  
enough warning that the code cache is getting full to start  
reclamation before you run out.  It's basically a GC policy problem  
though we can always simply delay compilation until some storage has  
been reclaimed.  A hiccup near the transition would still be better  
than our current policy of simply stopping compilation.

For the first part, I'd always imagined we end up with some strategy  
where we marked methods as speculatively not entrant and if they  
weren't used within some time period we'd transition them fully to not  
entrant and reclaim the nmethod storage.  One problem with this is the  
repatching of the nmethod back into a normal state.  I guess if it's  
done at a safepoint it wouldn't be a big deal.

On the other hand it occurred to me today that we could rely on the  
same guarantees that the nmethod sweeper uses to do this without  
requiring patching.  There are only two ways to get into compiled  
code.  You either grab the nmethod entry point from the methodOop you  
are invoking, which is how interpreter invokes and polymorphic  
dispatch work, or you have a CompiledIC that jumps you there directly  
which is the normal way compiled code works.  If you break both of  
these kinds of links then an nmethod can no longer be invoked.  This  
is essentially how making an nmethod not entrant works but since the  
cleaning of inline caches is done at a safepoint we also patch the  
entry point of the nmethod to dispatch back into the call resolution  
code.  This allows the cleaning of the CompiledICs to be done lazily  
because any caller that still references the nmethod will still end up  
back in the resolution code.

I think we could use the same strategy for identifying unused  
nmethods.  We simply break the reference from the methodOop to the  
nmethod and start a cache cleaning but without patching the entry  
point.  If we ever invoke those nmethods again we'll end up going  
through the call resolution code where we can detect that the nmethod  
is really being used and simply restore the nmethod and continue using  
it.  If it's never referenced within some time period the nmethod can  
simply be freed because we know that it's clean.

As far as policy goes we could simply mark every nmethod this way when  
we get into trouble but I think we'd want a more GC like policy that  
would begin identifying nmethods once the code cache started filling  
up and reclaim them before we actually run out.

tom

On Oct 12, 2009, at 9:53 AM, Eric Caspole wrote:

> We discovered some app servers do not have really well behaved class  
> loader schemes, so that in a long running process with multiple re- 
> deployments of the same web app, the old stuff never gets unloaded  
> and the first thing to run out of space in this case was the code  
> cache. In this event the compiler shuts itself off and does not turn  
> itself on regardless of what happens in later GC/sweep cycles.
>
> We have been wondering what it would take to do some kind of code  
> cache unloading so long running applications like this won't end up  
> having the later redeployments running interpreter-only. Many users  
> do not even know when this has happened and may or may not notice a  
> gradual mysterious slowdown until they just restart the process.
>
> In various discussions ideas have popped up from marking some amount  
> of existing nmethods non-entrant so they will get unloaded in the  
> existing code cache, to more elaborate reallocation schemes for the  
> whole code cache. If necessary, a short term slowdown before getting  
> back to having all the hot methods recompiled seems better than  
> restarting the process. Perhaps there could be some JMX notification  
> for this situation.
>
> One idea that came out of our testing was the idea of a current  
> working set of hot methods. We saw that if enough space could be  
> regained in the code cache, the program's normal operation would  
> require recompiles only of the current hot method set, which is  
> hopefully a lot smaller than the whole code cache size. Then it  
> would quickly resume normal operation, and only in the event of one  
> or more web app redeployments as mentioned above would the code  
> cache require another flush operation, hopefully days or weeks later.
>
> Lastly, it is probably desirable to have a fallback plan of giving  
> up and shutting off the compiler if the flush cycles are happening  
> too often, for example if the hot method working set size is too  
> close to the whole code cache size, and the application performance  
> won't be any worse than it gets today.
>
> We'd like to hear if anyone else has a strong opinion or great idea  
> on this topic, and what corner case did we not think of. I remember  
> a wise crack about this topic at the JVM Languages Summit a few  
> weeks ago so it seems someone out there is thinking about it.
>
> Thanks,
> Eric
>
>