RFR(S): 8029873: compiler/uncommontrap/TestStackBangRbp.java crashes with SIGSEGV

[Christian Thalinger]

Looks good.

On Jan 6, 2014, at 6:04 AM, Roland Westrelin <roland.westrelin at oracle.com> wrote:

> http://cr.openjdk.java.net/~roland/8029873/webrev.00/
> 
> Class loading from the uncommon trap blob calls java code which may trigger a stack overflow. If the overflow happens right before the java code returns to the runtime, the stack is left unguarded when we return to the uncommon trap blob. The stack banging in the uncommon trap blob can trigger a new stack overflow and because the stack is not guarded we crash. The fix consists in reguarding the stack after the class loading if needed (deoptimization.cpp).
> 
> The test case doesn’t reproduce this problem (I couldn’t get a test case to do it reliably). It triggers another similar problem. A stack overflow can be propagated to a compiled method while the stack is still unguarded. If the compiled method is marked for deoptimization then we go directly go to the deopt blob. If stack banging in the deopt blob triggers a new stack overflow we crash for the same reason as above. The fix is to reguard the stack before we go to the uncommon blob (sharedRuntime.cpp).
> 
> Roland.
>