RFR: 8296469: Instrument VMError::report with reentrant iteration step for register and stack printing [v9]

[Axel Boldt-Christmas]

On Fri, 5 May 2023 11:07:22 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

> It is certainly useful. I mainly regret the added complexity.
> 
> I wonder whether we need the stack headroom probing. AFAICS you limit the number of reattempts, maybe that's already enough. In earlier iterations of this patch, there were more reattempts possible.

In the very first iteration it the limits were the number of iterations the step required. After the initial discussions I added a lower per step and global limit.

I think in that iteration the total number of possible reattempts were the same as now. The previous implementation had a (customisable) per step and global reentry limit. Which by default was set to four per step and eight in total. 

The current iteration has the steps hard coded, so 3 + 3 + 2 which is still eight by default. 

But the customisable nature made more complex. The thinking was that this implementation that hard code the default values was a good compromise. 

Maybe the stack depth was only relevant when the number reattempts was `#printable registers + 8`

The stack depth checks can be removed if it makes this more palatable, it just does not seem worth printing some extra registers if the rest of the hs_err file printing failes due to a stack overflow.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/11017#issuecomment-1536151531