RFR: 8286104: use aggressive liveness for unstable_if traps [v2]

[Xin Liu]

On Thu, 19 May 2022 07:16:53 GMT, Tobias Hartmann <thartmann at openjdk.org> wrote:

>> Xin Liu has updated the pull request incrementally with 11 additional commits since the last revision:
>> 
>>  - revert code change from 1st revision.
>>  - Merge branch 'JDK-8276998' into JDK-8286104
>>  - rule out if a If nodes has 2 branches of unstable_if trap.
>>  - change the flag to diagnostic.
>>  - add sanity check for operands if bc is if_acmp_eq/ne and ifnull/nonnull
>>  - fix release build
>>  - update unstable_if after igvn.
>>  - adjust unstable_if after fold_compares
>>  - disable comparison_folding temporarily.
>>    
>>    This feature not only folds two CMPI but also merge two uncommon_traps.
>>    it uses the dominating uncommon_trap and revaluate the two if in
>>    interpreter.  currently, aggressiveliveness can't work for that.
>>  - retain bci for unstable_if
>>  - ... and 1 more: https://git.openjdk.java.net/jdk/compare/2c38b87b...2f047457
>
> I ran this through some quick testing and `test/hotspot/jtreg/compiler/rangechecks/TestExplicitRangeChecks.java` fails:
> 
> java.lang.reflect.InvocationTargetException
> 	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:116)
> 	at java.base/java.lang.reflect.Method.invoke(Method.java:578)
> 	at compiler.rangechecks.TestExplicitRangeChecks.doTest(TestExplicitRangeChecks.java:441)
> 	at compiler.rangechecks.TestExplicitRangeChecks.main(TestExplicitRangeChecks.java:518)
> 	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
> 	at java.base/java.lang.reflect.Method.invoke(Method.java:578)
> 	at com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapper.java:127)
> 	at java.base/java.lang.Thread.run(Thread.java:1585)
> Caused by: java.lang.NullPointerException: Cannot read the array length because "<parameter2>" is null
> 	at compiler.rangechecks.TestExplicitRangeChecks.test3_2(TestExplicitRangeChecks.java:113)
> 	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
> 	... 7 more

@TobiHartmann 
I studied that failure. It's still from the conflict with 'fold-compares'.  Some compare nodes look like range check, so c2 takes a record and postpone 'fold-compares' transformation after loop optimizations. I killed locals of the uncommon_trap prematurely.  

My new solution: disqualify the uncommon_trap  when `has_only_uncommon_traps()` is about to return true. It returns true when 2  if nodes both have uncommon traps and c2 will fuse them. I disqualify the dominating one and the other one will be dead.

-------------

PR: https://git.openjdk.java.net/jdk/pull/8545