RFR 8009302: JVM crash on infinite recursion on Appkit Thread

[David Holmes]

A big thumbs up from me Gerard! The investigation you did here was 
outstanding!

I nominate you to look into the stackoverflow issues reported against 
Linux ;-)

Thanks,
David

On 5/06/2013 4:34 AM, Gerard Ziemski wrote:
> hi guys,
>
> Here is a trivial fix (workaround) for a particular XNU (Mac OS X
> kernel) behavior causing JavaVM problems.
>
> Explanation:
>
> There is a case where XNU will map SIGBUS into SIGSEGV, which only
> happens on main thread (secondary threads receive the original SIGBUS)
> due to the way XNU determines stack address and its size. To complicate
> the issue, XNU will only deliver that converted SIGSEGV signal if the
> BSD signal catcher uses an alternate stack.
>
> Since JavaVM BSD signal catcher does not use alt stack, we end up
> missing the signal and the Java process results in a native crash which
> is then presented to the user via Mac CrashReporter app.
>
> We do not want to use an actual alternate stack, which among other
> things would result in an increased RAM usage and possibly result in a
> regression, so we use a workaround, where we declare we will use an
> alternate stack, without actually creating one.
>
> This workaround is only for Mac (ie. we use #ifdef __APPLE__) and it
> works because we install our own guard pages, and that's where the
> faulting address occurs, so we are assured that our stack does have
> enough room to handle the signal by the existing JavaVM code, which
> removes the guard pages before handling the signal.
>
> I have filed an issue with Apple (id: 14047928) to fix this inconsistent
> (wrt threads) behavior in XNU, and wrote native tests that check for
> this particular behavior, which will catch any future change, should
> Apple ever elect to fix it.
>
>
> Testing:
>
> - the issue's own use case
> - vm.quick.testlist
> - vm.signal.testlist
> - nsk.stack.testlist
> - JPRT
> - JCK lang
> - JCK vm
>
>
> References:
>
> Webrev: http://cr.openjdk.java.net/~hseigel/bug_8009302/
>
> OpenJDK bug  http://bugs.sun.com/view_bug.do?bug_id=8009302
>
> JBS bug: https://jbs.oracle.com/bugs/browse/JDK-8009302
>