RFR: 8271600: C2: CheckCastPP which should closely follow Allocate is sunk of a loop [v2]

[Vladimir Ivanov]

On Mon, 23 Aug 2021 13:16:49 GMT, Vladimir Ivanov <vlivanov at openjdk.org> wrote:

>> `PhaseIdealLoop::try_sink_out_of_loop()` tries to move nodes outside a loop. It doesn't work well for cast nodes which accompany `Allocate` nodes:  the cast reifies the effect instance initialization and turns a raw pointer into an oop. Once the cast node doesn't immediately follow the corresponding `Initialize`, it creates a window where the raw pointer is observed. And it can become so wide that the raw pointer crosses a safepoint and has to be recorded in the oop map (that's the case when the reported assertion failure happens). 
>> 
>> Proposed fix is to keep such cast nodes intact. Moreover, I propose to disable the optimization for all flavors of cast nodes. 
>> Cast nodes usually accompany runtime checks and I don't see much benefit in moving the cast node away while leaving the runtime check inside the loop.  
>> 
>> Testing: hs-tier1 - hs-tier6
>
> Vladimir Ivanov has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Limit the fix to raw-to-oop casts only.

Ok, I'm fine with limiting the fix for raw-to-oop casts only.

> The change you propose would then prevent sinking out of the outer loop.

I thought that the code at [loopopts.cpp#L1481](https://github.com/openjdk/jdk/blob/master/src/hotspot/share/opto/loopopts.cpp#L1481) would handle it. Anyway.

-------------

PR: https://git.openjdk.java.net/jdk/pull/5199