Integrated: 8257594: C2 compiled checkcast of non-null object triggers endless deoptimization/recompilation cycle

[Tobias Hartmann]

On Wed, 2 Dec 2020 15:22:57 GMT, Tobias Hartmann <thartmann at openjdk.org> wrote:

> When C2 determines at compile time that there is no subtype relation between the objects involved in a checkcast, it inserts a null assert because the checkcast can only pass if the value is null. This will trigger deoptimization and recompilation if the object is non-null. However, we can do better here if we already know that the object is non-null (i.e. the check always fails) or if we hit `too_many_traps_or_recompiles` (which suggests that the object is often null) to avoid an endless deoptimization/recompilation cycle until we hit the `PerMethodRecompilationCutoff` and starve at lower tiers.
> 
> In the former case we can simply throw right away and in the latter case we should fall back to a full check in case there is an exception handler in compiled code that handles the failing (non-null) case. That way we avoid deoptimization.
> 
> I've spotted this with our test framework for inline types in project Valhalla, while investigating an intermittent failure where several test methods suddenly became non-compilable at tier 4.
> 
> Thanks,
> Tobias

This pull request has now been integrated.

Changeset: 129c3770
Author:    Tobias Hartmann <thartmann at openjdk.org>
URL:       https://git.openjdk.java.net/jdk/commit/129c3770
Stats:     133 lines in 4 files changed: 131 ins; 0 del; 2 mod

8257594: C2 compiled checkcast of non-null object triggers endless deoptimization/recompilation cycle

Reviewed-by: roland, vlivanov

-------------

PR: https://git.openjdk.java.net/jdk/pull/1566