RFR: 8358781: C2 fails with assert "bad profile data type" when TypeProfileCasts is disabled

[Damon Fenacci]

On Tue, 5 Aug 2025 10:40:19 GMT, Saranya Natarajan <snatarajan at openjdk.org> wrote:

> **Issue**
> An error, `assert(data->is_ReceiverTypeData()) failed: bad profile data type`, is encountered during C2 compilation due to bad profile data. This occurs when the code is compiled with `TypeProfileCasts` option disabled. 
> 
> **Analysis** 
> The assertion failure occurs in `record_profiled_receiver_for_speculation` that analyzes the profiling information in the method data to determine whether a null value has been observed in the `instanceof` operation. This information is encoded in the `BitData` during profiling. When the method identifies that a null has been seen, it proceeds to inspect the associated `ReceiverTypeData` to see if the  type check is always performed against null. However, in this scenario, the incoming profiling data is of type `BitData` rather than `ReceiverTypeData`, leading to the assertion failure. 
> 
> The profiling information for null seen for operations `aastore`, `instanceof`, and `checkcast` is recorded by the method `profile_null_seen `(in` src/hotspot/cpu/x86/templateTable_x86.cpp `). On investigating this method, it can be observed that the method data pointer is not updated for `VirtualCallData` (which is a subclass of `ReceiverTypeData`) when the `TypeProfileCasts` option is disabled. 
> 
> **Solution** 
> My proposal is to inspect the `ReceiverTypeData` in function `record_profiled_receiver_for_speculation` only if `TypeProfileCasts` is enabled (this is based on the fact that the relevant method data pointer is not updated when `TypeProfileCasts` is disabled).
> 
> **Question to reviewers** 
> Do you think this is a reasonable fix ?
> 
> **Testing**
> GitHub Actions
> tier1 to tier3 on windows-x64, linux-x64, linux-aarch64, macosx-x64, and macosx-aarch64.

Thanks for fixing this @sarannat. Apart from the missing regression test already mentioned by @mhaessig the fix looks good to me.
Just a quick question: did you try to run some testing with `-XX:-TypeProfileCasts`?

-------------

Marked as reviewed by dfenacci (Committer).

PR Review: https://git.openjdk.org/jdk/pull/26640#pullrequestreview-3099980060