Sensitivity of HotSpot to late-breaking news

[Charles Oliver Nutter]

JRuby normally interprets all code on startup. This is partially for 
startup performance reasons, but also because a large percentage of Ruby 
code is either only used once or is generated at runtime. So we have a 
simple call-count threshold at which we will JIT compile Ruby code into 
Java bytecode. Currently, the threshold is 50 invocations.

In this email I'm seeking to understand the effect of late-breaking 
changes on hotspot's profiled optimizations. Specifically, since JRuby's 
typical execution model means that basically *all* methods will be 
following the interpreted path for a while and then switch to a compiled 
path, how severely could that be damaging HotSpot's optimizations?

We have seen, for example, microbenchmarks that run great for the first 
50 iterations. After 50 iterations, the benchmark library itself starts 
to JIT, and performance takes a brief drop. But it never fully recovers, 
even though many (all) subsequent invocations do not further change the 
type profile at whatever location led to the deopt in the first place.

Essentially it seems like we need a way to tell hotspot "forget all 
that, I'm really, truly switching to this new type forever and ever", 
allowing the same monomorphic optimizations to be applied to the new type.

Is this possible with current HotSpot? Do you have recommendations for 
how best to avoid this effect (like using a boolean to switch from 
interpreted to compiled, rather than swapping implementations?). I'm 
concerned about it because I would also now like to start doing active 
replacement of code at runtime in response to JRuby-level profiling, and 
if replacing one implementation with another (possibly permanently) 
damages long-term performance it will certainly be less attractive.

- Charlie