Request Review: 6950075: nmethod sweeper should operate concurrently

[Eric Caspole]

In the current JVM, the nmethod sweeper runs during each safepoint,
and may return early if there is no work to do. It periodically scans
the thread stacks to determine if not-entrant nmethods have no
lingering active stack frames and can be converted to zombies. It
then walks the code cache promoting not-entrant nmethods to zombies
if possible and deleting the zombies. It walks a fraction of the code
cache in each turn, set by NmethodSweepFraction, which is 4 by
default. When the whole code cache is traversed and no progress can
be made, the sweeper will go idle until another nmethod becomes not-
entrant, starting the process over again.

In applications with a lot of compiled code, walking the code cache
can take many milliseconds, contributing to long safepoint pause
times, reducing application throughput. This change makes part of the
sweeper work concurrent to reduce the safepoint pause time.

In this webrev, the stack scans continue to happen at safepoint time
but the code cache traversal is concurrent with the application. The
traversal work runs on the compiler threads where they will check to
sweep each time passing through the compile queue. They may take the
code cache and stub related locks as necessary to do the sweeping
safely. Only one thread will sweep at a time and it preserves the
existing way of traversing in chunks by NmethodSweepFraction.

There are a few adjustments to the checks regarding the code cache
unloading feature to accommodate these changes. Also I added new
convenience methods such as CodeCache::next_nmethod() for simple
traversal.

I worked out this design with Tom Rodriguez over the last month or  
so. We'd like some other eyes on it.

The webrev is at
http://cr.openjdk.java.net/~ecaspole/concswpr/webrev.09/webrev/

Thanks,
Eric