RFR (S) 7014552: gc/lock/jni/jnilockXXX works too slow on 1-processor machine

[Mikael Gerdin]

Hi

Please review the following change:

When running a stress test that is repeatedly blocking GC from 
triggering through the normal path we may get stuck in the allocation 
path and stay in the allocation loop forever.

In this case the problem seems to be that if some threads are repeatedly 
entering and leaving JNI critical regions and thereby stressing the 
GC_Locker an allocating thread may get stuck in the allocation loop and 
never realize that the pending allocation is impossible due to heap usage.

My suggested fix is to limit the amount of times we try to start GCs and 
then simply give up and return NULL and fail the allocation, thereby 
causing the caller to throw an OOME.

Note that every time the counter is incremented because we return from 
GC_Locker::stall_until_clear() the thread unlocking the GC_Locker has 
actually triggered a GC. So we're not giving up without a fight.

Public bug: http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7014552
JIRA:       https://jbs.oracle.com/bugs/browse/JDK-7014552
Webrev:     http://cr.openjdk.java.net/~mgerdin/7014552/webrev.0/

Thanks
/Mikael