Valhalla EG meeting notes Sep 12 2018

[Tobias Hartmann]

[Switching from valhalla-spec-experts to valhalla-dev]

Just wanted to add my 2 cents to the JIT part of this discussion:

On 13.09.2018 00:22, Karen Kinnear wrote:
> Frederic: LW1 uses NO boxes today. JIT could not optimize boxes, can we consider a model without boxes?

+1

> Brian: MVT, L&Q signatures were messy with boxes. With LWorld all those problems go away.
> Frederic: Question of the number of types in the vm.
> Remi: Two types in the language level and 1 in VM.
> Karen: Goal of this exercise is:
>    1) user model requirements to support erased generics - requires null and null-free references to value types
>    2) JIT optimizations in general for value types (not for erased generics, but in general code and in future in reified generics) - depend on null-free guarantees.
> Our goal: minimal user model disruption for maximal JIT optimization.

As explained below, for the JIT it would be optimal to be able to statically (i.e. at compile time)
distinguish between nullable and null-free value types. We could then emit highly optimized code for
null-free value types and fall back to java.lang.Object performance (or even better) for nullable
value types.

> The way I read your email Remi - I thought you had user model disruption, but no information passed to the JIT, so no optimization benefit.
> Remi: If inlining, the JIT has enough information.

Yes but even with aggressive inlining, we still need null checks/filtering at the "boundaries" to be
able to optimize (scalarize) nullable value types:
- Method entry with value type arguments
- Calls of methods returning a value type
- Array stores/loads
- Field stores/loads
- Loading a value type with a nullable (non-flattenable) field
- Checkcast to a value tyoe
- When inlining method handle intrinsics through linkTo and casting Object arguments to value type
- OSR entry with a live value type
- Every place where we can see constant NULL for a value type in the bytecodes
- Some intrinsics

> Frederic: Actually with field and method signatures it makes a huge difference in potential JIT optimizations.

Yes, it would make above null filtering unnecessary for null-free value types.

> Remi: Erased generics should have ok performance
> Frederic: We are talking about performance without generics - we want full optimization there.
> Remi: what happens in LW1 if you send null to a value type generic parameter?
> Frederic: LW1 supports nullable value types. We want to guarantee/enforce null-free vs. nullable distinction in the vm.

> Remi: there are two kinds of entry points in the JIT’d code
> Frederic: 2: i2c which does null checks and if null calls the interpreter, c2c - disallows nulls.
> editor’s note: did you get the implication - if we see a null, you are stuck in the interpreter today because we all rely on dynamic checks.

Yes, that's an important point.

> Remi: For the vm, if you have a real nullable VT JIT can optimize/deopt/reopt
> Frederic: this is brittle, lots of work, and uncertain

That's what we currently have with LW1. Although the language exposure is limited by javac, null
value types are *fully* supported in the VM/JIT but with a huge performance impact. We deoptimize
when encountering NULL but do not attempt to re-compile without scalarization. We could do that, but
that would mean that whenever you (accidentally) introduce NULL into your well-written, highly
optimized value type code, performance will drop significantly and stay at that level.

Of course, there are ways to optimize this even without null-free value types. For example by
profiling and speculation on nullness. Or by having two compiled versions of the same method, one
that supports nullable value types by passing them as pointers (no deoptimization) and one that
scalarizes null-free value types to get peek performance.

But as Frederic mentioned, these approaches are limited, complex and the gain is uncertain. It's
maybe similar to escape analysis: We might be able to improve performance under certain conditions
but it's very limited. I think the whole point of value types is performance, so we should try to
get that right.

To summarize: If we just need to support null value types and are fine with null screwing up
performance and having weird side effects, we are basically done today. If null value types need to
perform well (i.e. similar to j.l.Object), optimally we would need to be able to statically
distinguish between nullable and null-free value types.

> Remi: VM does not want nullable value types
> Frederic: VM wants to be able to distinguish null-free vs. nullable value types, so 
> for null-free we can optimize like qtypes and fo nullable, we can get back to Object performance.

Yes, exactly.

Best regards,
Tobias