RFR: 8330157: C2: Add a stress flag for bailouts [v10]

[Daniel Skantz]

On Mon, 23 Sep 2024 06:21:18 GMT, Daniel Skantz <duke at openjdk.org> wrote:

>> This patch adds a diagnostic/stress flag for C2 bailouts. It can be used to support testing of existing bailouts to prevent issues like [JDK-8318445](https://bugs.openjdk.org/browse/JDK-8318445), and can test for issues only seen at runtime such as [JDK-8326376](https://bugs.openjdk.org/browse/JDK-8326376). It can also be useful if we want to add more bailouts ([JDK-8318900](https://bugs.openjdk.org/browse/JDK-8318900)).
>> 
>> We check two invariants.
>> a) Bailouts should be successful starting from any given `failing()` check.
>> b) The VM should not record a bailout when one is pending (in which case we have continued to optimize for too long).
>> 
>> a), b) are checked by randomly starting a bailout at calls to `failing()` with a user-given probability.
>> 
>> The added flag should not have any effect in debug mode.
>> 
>> Testing:
>> 
>> T1-5, with flag and without it. We want to check that this does not cause any test failures without the flag set, and no unexpected failures with it. Tests failing because of timeout or because an error is printed to output when compilation fails can be expected in some cases.
>
> Daniel Skantz has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 40 commits:
> 
>  - phrasing
>  - nextInt
>  - +1 case
>  - words
>  - Tweak CtwRunner.java; debug only bool flag
>  - merge
>  - +1 whitespace
>  - tweak requires
>  - rename secondary flag v2
>  - propagate external flags to test
>  - ... and 30 more: https://git.openjdk.org/jdk/compare/10050a72...9ab95aa6

src/hotspot/share/opto/compile.cpp line 1124:

> 1122:  */
> 1123: StartNode* Compile::start() const {
> 1124:   assert (!failing_internal() || C->failure_is_artificial(), "Must not have pending failure. Reason is: %s", failure_reason());

Having the stress mode in debug builds requires weakening asserts since debug builds assert these paths are not taken.

test/hotspot/jtreg/testlibrary/ctw/src/sun/hotspot/tools/ctw/CtwRunner.java line 309:

> 307:                 "-XX:StressSeed=" + rng.nextInt(Integer.MAX_VALUE)));
> 308: 
> 309:         // Use this stress mode 10% of the time as it could make some long-running compilations likely to abort.

I apply this stress mode randomly as it could hypothetically hide an infinite compilation bug if used unconditionally.
If it is later used in another test suite we might need similar randomization.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/19646#discussion_r1771061037
PR Review Comment: https://git.openjdk.org/jdk/pull/19646#discussion_r1771061584