RFR(L): JDK-8032463 VirtualDispatch test timeout with DeoptimizeALot

[Igor Veresov]

The current implementation of the code cache flusher tries to determine the working set of methods by doing periodic stack walks during safepoints. Methods that are seen on the stack are considered to be used by the program and are excluded from the set of flush candidates. Such sampling is a great zero-overhead approach to collect the working set of super-hot methods without instrumentation. However it doesn’t work that good for flat profiles, which the test in the bug report happens to expose (thousands of methods calls sequentially one after another in the loop). The solution I ended up implementing uses conservative on-demand instrumentation to obtain information about the method usage. The algorithms is as follows:

1. Initially a method is compiled as usual (no instrumentation, no overhead).
2. When the sampling (stack-walking) scheme fails to detect activity we flush the method (again, as usual). However before the flush we set the age counter (an int added to MethodCounters) to a value that instructs the compilers to instrument the code.
3. If we ever request to compile this method again the aging code is inserted, which decrements the counter.
4. The value of the counter is then used by the sweeper to determine is the method is in fact used.
5. If the counter reaches zero before the sweeper is able examine and reset the counter we deopt and recompile the method without the counters, basically switching it back to use the sampling scheme again. 

Webrev: http://cr.openjdk.java.net/~iveresov/8032463/webrev.00/

Testing: 
- jprt,
- aurora-perf No performance effects with the default code cache, since we have enough headroom and the flusher is not exercised that much. No statistically significant effects after warmup (for spec benchmarks) in stress mode (-XX:+StressCodeAging) either, that is if all methods are compiled initially with counters deopt and switch to sampling later after 100000 invocations.

igor