RFR: 8296469: Instrument VMError::report with reentrant iteration step for register and stack printing

[Thomas Stuefe]

On Mon, 7 Nov 2022 13:24:26 GMT, Axel Boldt-Christmas <aboldtch at openjdk.org> wrote:

> Add reentrant step logic to VMError::report with an inner loop which enable the logic to recover at every step of the iteration.
> 
> Before this change, if printing one register/stack position crashes then no more registers/stack positions will be printed.
> 
> After this change even if the VM is unstable and some registers print_location crashes the hs_err printing will recover and keep attempting to print the rest of the registers or stack values.
> 
> Enables the following
> ```C++
> REENTRANT_STEP_IF("printing register info", _verbose && _context && _thread && Universe::is_fully_initialized())
>   os::print_register_info_header(st, _context);
> 
>   REENTRANT_LOOP_START(os::print_nth_register_info_max_index())
>     // decode register contents if possible
>     ResourceMark rm(_thread);
>     os::print_nth_register_info(st, REENTRANT_ITERATION_STEP, _context);
>   REENTRANT_LOOP_END
> 
>   st->cr();
> 
> 
> Testing: tier 1 and compiled Linux-x64/aarch64, MacOS-x64/aarch64, Windows x64 and cross-compiled Linux-x86/riscv/arm/ppc/s390x (GHA and some local)

Maybe a pragmatic combination of approaches would work, e.g. allow a limited number of recursive attempts for register printing, combined with looking at the printer. I would hope that most cases trip over a small number of issues and that we can solve them without wrapping every load with SafeFetch. That would admittedly be very ugly.

I think we do this in places already, e.g. checking Klass* for validity (needs to be in committed class space, needs to be aligned, needs to point to mapped memory yada yada)

-------------

PR: https://git.openjdk.org/jdk/pull/11017