RFR: 8284828: Use `os::ThreadCrashProtection` to protect AsyncGetCallTrace from crashing

[Thomas Stuefe]

On Wed, 13 Apr 2022 14:53:54 GMT, Johannes Bechberger <duke at openjdk.java.net> wrote:

> Move the AsyncGetCallTrace method implementation into a separate method and wrap its call in non-assert compilation mode in `os::ThreadCrashProtection` like it is done in [JFR](https://github.com/openjdk/jdk/blob/965ea8d9cd29aee41ba2b1b0b0c67bb67eca22dd/src/hotspot/share/jfr/periodic/sampling/jfrThreadSampler.cpp#L165).
> This prevents AsyncGetCallTrace from crashing on segmentation faults (but not on `guarantee`s).
> 
> If a crash is observed, then the `num_frames` field of the trace is set to `ticks_unknown_state` (-7) to signal a state that cannot be properly handled. `ticks_unknown_state` is currently also used for signaling unknown thread states but this should not be a problem, as the semantic is the same. If `num_frames` already has an error code then this error code is not changed. This helps to distinguish between errors in walking threads in Java and non-Java mode, as `num_frames` is set there before the walking to the appropriate error code.
> 
> _Thanks for @tstuefe for suggesting this._

src/hotspot/share/prims/forte.cpp line 508:

> 506: 
> 507: 
> 508: void asyncGetCallTraceImpl(ASGCT_CallTrace *trace, jint depth, void* ucontext) {

make it static? No need to export this.

src/hotspot/share/prims/forte.cpp line 669:

> 667: JNIEXPORT
> 668: void AsyncGetCallTrace(ASGCT_CallTrace *trace, jint depth, void* ucontext) {
> 669: #ifndef ASSERT

Since ThreadCrashProtection uses Thread::current, here I would bail for Thread::current_or_null_safe() == NULL

src/hotspot/share/prims/forte.cpp line 674:

> 672:   os::ThreadCrashProtection crash_protection;
> 673:   if (!crash_protection.call(cb)) {
> 674:     if (trace->num_frames < -10 || trace->num_frames >= 0) {

Where does this -10 come from?

-------------

PR: https://git.openjdk.java.net/jdk/pull/8225