RFR(XS): 8043070: nmethod::verify_interrupt_point() shouldn't enter safepoint

[Igor Veresov]

So, I went back to this bug and here is the actual root cause of the problem... I was running stress tests with  -XX:MinPassesBeforeFlush=0 and a very small code cache, which created a race between the method installation (in ciEnv::register_method()) and the flusher. Methods were flushed right after we released the CodeCache_lock in nmethod::new_nmethod(), and that’s the reason the verification thought that the installation has finished since the method wasn’t in use. It’s also btw pretty dangerous to do an nmethod state transition from in_use to not_entrant before the code pointer is installed - we don’t clean it up later when we transition from not_entrant to zombie (and we can’t).

Anyways, the solution is pretty straightforward, we must not flush an nmethod before it’s fully installed (that is until the code pointer is published).
Webrev: http://cr.openjdk.java.net/~iveresov/8043070/webrev.02/

Thanks,
igor