RFR: 8282306: os::is_first_C_frame(frame*) crashes on invalid link access [v4]

[Johannes Bechberger]

On Thu, 24 Feb 2022 14:26:39 GMT, Johannes Bechberger <duke at openjdk.java.net> wrote:

>> This PR introduces a new method `can_access_link` into the frame class to check the accessibility of the link information. It furthermore adds a new `os::is_first_C_frame(frame*, Thread*)` that uses the `can_access_link` method
>> and the passed thread object to check the validity of frame pointer, stack pointer, sender frame pointer and sender stack pointer. This should reduce the possibilities for crashes.
>
> Johannes Bechberger has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Introduce frame::link_or_null()

I changed it again, introducing "frame::link_or_null()" that is the safe version of "frame::link()".

> About a test, I agree, that would be nice. But one would have to "fake" an invalid stack. Maybe a new error reporting test where one deliberately overwrites portions of the stack and then tries to print the stack. However, I imagine things could be brittle, because the OS may catch a stack overwrite first. It's not totally trivial, maybe something for a separate RFE?

I think tests would be nice but also quite difficult. A simple test would be to allocate a frame with zero values for all entries and check that `os::is_first_C_frame` returns true and that `frame::link_or_null()` returns also null. Then the same with a good frame (pointing to sensible values).

-------------

PR: https://git.openjdk.java.net/jdk/pull/7591