Lazy statics (was: Feedback / query on jextract for Windows 10)

[John Rose]

On Feb 3, 2021, at 11:01 AM, Brian Goetz <brian.goetz at oracle.com> wrote:
> 
>>> Reframing as caching makes it more obvious that we have to be clear
>>> about what guarantees we are offering, especially in the instance
>>> case; are we guaranteeing to execute the method no more than once, or
>>> is it OK for multiple threads to race to initialize the cached
>>> value?  What visibility guarantees do we make?
>> So, it seems to me that caching is a bit clearer in setting
>> expectations when it comes to migrating away from a "final static"
>> field, as well as capturing the throwy nature of these things.

(Bikeshed:  We could call them “once-only” methods as well as
“cached” or “lazy”.  Seems to me that some languages have
“once” constructs.)

A once-only/cached/lazy method can be given a throws clause to
document exceptions, including checked exceptions.  Static fields
can’t do this as well.

A once-only/cached/lazy method can be marked synchronized.
If its body is then compiled as an ldc of a condy, then you get
strong guarantees that the evaluation of the constant value is
done under mutual exclusion.

So both throws and synchronization work “out of the box”
for once-only/cached/lazy methods.

In the non-synchronized case, I think the condy/indy rules
for resolving races are just fine.  They say that racing threads
might redundantly compute the constant value, but only
one thread is picked as the winner (using CAS or an equivalent)
and all threads see the winning value.  If you need to prevent
redundant computation, just add synchronization.  The JIT
can undertake to optimize calls to such synchronized methods,
since the body of the methods has zero side effects, after the
condy is first resolved.  Probably it already does this right,
since empty synch. blocks are elided.

The compilation strategy requires no new VM features.
You need a source modifier of some sort on the once-only/cached/lazy
method M, and then M’s body gets desugared into a separate private
static method P, with M containing “ldc condy P; areturn;”

Seems like a clean story for the JVM and the user.  It’s odd
that it only applies to zero-argument methods, but it’s a plus
that they *are* real methods, not some new kind of API point.

As a separate move, the ideas could be applied to static finals.
I think the adjustments would as follows:

- The initializer expression would be packaged into a private
method body, as with M and P above.  (Note there is no touch
to <clinit>.)

- Synchronization would not be directly available (but
could be obtained by means of a private once-only P).

- Exceptions would not be allowed.  I think even the normal
RuntimeExceptions (like NPE) would want to be repackaged
as LinkageErrors, which is what condy does anyway.

- The JVM would need special features to (a) mark which
condy was bound to the static final, and (b) to add an
extra step to resolution of getstatic bytecodes.

- Resolution already includes arbitrary computation,
but by the time that a client sees the resolved field, it
is indistinguishable from a normal, eagerly initialized
static field.  Thus, the impact on the getstatic instruction
is very low; it’s a variant case of what the JVM already
does for today’s idiom of putting a single field into
a holder class, to get lazy initialization.

- From a VM point of view, this variation (the lazy or
once-only or cached static final field) simply provides
a way to give a linkable name to a condy slot.  (With
access checking as a bonus.)  The once-only method
which wraps a condy is, similarly, a way to give a
linkable *method-kinded* name to a condy slot,
but the field seems (from a VM POV) a better fit.

- Here’s one reason we might wish to do *both* lazy
fields *and* lazy/once-only methods:  The JDK and
other libraries are full of API points which surface
global constants, and if we are to persuade programmers
to re-implement them as lazies, we had better not
require the programmers to change their APIs.
Today it’s hard to tell programmers, “just recode
your API points in the one-field-per-class design
pattern”, even for private constants.  It will help
them recode if we tell them, “just recode them
as lazy fields or methods, and don’t worry about
binary compatibility”.

BTW, AOT projects like our “aotc” found that
their initialization sequencing causes headaches
an AOT optimizer, because it must schedule the
initialization of *all* statics of a class the first time
*any* static of a class.  A lazy has a more manageable
set of effects:  The AOT engine simply has to ensure
that the one constant it is accessing is initialized,
and that can be separated and rescheduled relative
to whatever side-effects might be happening in the
<clinit> of the target class.

I had not seriously considered the cached/once-only
method design until today, because the use cases
I was looking at are retrofitting old static-finals,
not new code.  But that was short-sighted of me.
Based on this discussion, I think we should do the
method version of things (with no JVM changes)
first, but keep open the option for applying the
techniques to static final fields (with JVM changes
(a) and (b) above), as a tool for retrofitting existing
APIs that include fields (as many do).

As a cleanup, we should simplify our source base
to *remove* instances of the one-field-one-class
pattern (for lazy initialization), moving those
magic fields into the most natural containing
class, and marking them lazy.

>> 
>> The point you raise about thread racing to init an instance constant is
>> a well taken one, and I think it's a pretty bad problem to have to
>> solve - of course we have the tools (atomic access, etc.) but seems
>> expensive-ish, and the performance model is probably not very
>> transparent from the client side.
>> 
> 
> Agreed on both points.  I think it is a slightly less magic user model, and gives the runtime a little more breathing room because rather than tying to a representation (there's a field called x that when read, should have this value) we're talking about what result we want to achieve (when this method is called multiple times, the result of a previous call is cached and reused).

That’s why I think we should do the methods first.
But there are reasons to do the fields also, since
plenty of APIs already use fields and can’t convert
to methods.

> With respect to racing, moving caching into the runtime means that we may need knobs for exactly what cache coherence semantics we want.  But, we do have modifiers like `volatile` and `synchronized` which could be pressed into service to distinguish between "best efforts, last write wins" and "only let one thread run this." (Which may, or may not, send the right message.)

If we use condy (which we should!) then we get “best efforts, first
write wins”, which is a pretty good model, better than “last write
wins”, since everybody sees one value.

— John