RFR: 8355488: Add stress mode for C2 loop peeling

[Marc Chevalier]

On Tue, 13 May 2025 10:36:55 GMT, Tobias Hartmann <thartmann at openjdk.org> wrote:

>> Adding a `StressLoopPeeling` dev flag that randomize peeling.
>> 
>> ## Semantics
>> 
>> For now, the direction I've taken is to randomly take a decision in case of peeling, otherwise, rely on existing heuristics.
>> 
>> This requires to distinguish two things:
>> - not inlining because it's not legal: see for instance
>>   ```cpp
>>   assert(cl->trip_count() > 0, "peeling a fully unrolled loop");
>>   ```
>>   in `PhaseIdealLoop::do_peeling`
>> - not inlining because it doesn't seem profitable.
>> 
>> Peeling loops without a good reason (not containing an exiting `If` whose condition is not a member of the loop) but without a concrete way to forbid it should always be allowed. Let's stress it!
>> 
>> Peeling too many times is not a great idea either. It uses a lot of memory, of nodes... Also, it may prevent other optimisations from kicking in. And what about interaction with future stress flags? Let's limit peeling: we give a fixed number of opportunities to peel before we give up on peeling for good. That is not the same as limiting the amount of peeling we do. Indeed, if we bound the number of times we say "yes, please, peel" given enough requests, we will always reach the bound. If we limit the number of requests, we have a more evenly distributed amount of peeling, between 0 and the bound.
>> 
>> I've tried without the bound: I couldn't find any bug without the bound that would not reproduce with the bound. It only save some legitimate memory problems. Without a bound on the number of peeling opportunities, hotspot eats a lot of memory, but all the allocations seems reasonable: it just seems we ask too much. We could limit the number of nodes, to prevent peeling before we reach the memory limit, but that would also hinder other optimizations and (future) stress flags.
>> 
>> 
>> 
>> ## The Flag
>> 
>> The flag is very specialized, unlike a `StressLoopOpts` would be. My idea so far is "let's see". My idea is that it's good to be able to enable stress optimizations selectively, and have a flag like `StressLoopOpts` that would turn them all: we could use the general one in testing, and the finer-grain ones when debugging. A reason for that is that I don't see a real use-case for stressing some features but not others (which would make the number of combinations explode): having (for instance) `+StressLoopUnrolling +StressLoopPeeling` would sometimes behave like `+StressLoopUnrolling -StressLoopPeeling`, and so it's not very useful to test the latter.
>> 
>> But once again: let'...
>
> src/hotspot/share/opto/compile.cpp line 666:
> 
>> 664:       _congraph(nullptr),
>> 665:       NOT_PRODUCT(_igv_printer(nullptr) COMMA)
>> 666:       NOT_PRODUCT(_peeling_rounds_of_node(comp_arena(), 8, 0, Pair<node_idx_t, uint>(0, 0)) COMMA)
> 
> `NOT_PRODUCT` means that it's also available in the optimized build but you only want/need it in debug.

Should I use `#ifdef ASSERT`? My thinking is that I want the code to be there whenever the flag is available. This is decided here:
https://github.com/openjdk/jdk/blob/a989245a2424d136f5d2a828eda666c3867b0f48/src/hotspot/share/runtime/flags/jvmFlag.cpp#L552-L556
(called from `JVMFlag::find_flag` with `return_flag == false`, from `Arguments::find_jvm_flag`, from `Arguments::parse_argument` from `Arguments::process_argument`) with
https://github.com/openjdk/jdk/blob/a989245a2424d136f5d2a828eda666c3867b0f48/src/hotspot/share/runtime/flags/jvmFlag.cpp#L60-L66

My thinking is that it's unintuitive to me to offer a flag that could have no effect. Why don't we want it in optimized non-product build? We can still stress-peel and even if we won't hit an assert, we could still observe unexpected behaviors or crashes. Does that make sense?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/25140#discussion_r2088982156