Some offhand questions

[John Rose]

On 23 Aug 2024, at 15:04, Cesar Soares Lucas wrote:

> Hello!
>
> I've a few questions that I'd like to ask your opinion about.
>
> - Signed Jars: As far as I understand, we currently don't include classes from signed jars in the CDS archive. What is the reason for that? I had the impression that being able to archive such classes would be important given that many .jars are signed?!

Those were good replies from Alan, Dan, and Ioi…

This is an example (one of many) of a possible work-item which we occasionally
discuss, and then put back onto the priority list, very low down.  In this case
we might escape ever implementing it, unless the feature is somehow found to be
important to users of the AOT cache.  (And, it survives into the future.)

We almost need a shorthand here for “yes we have discussed that feature,
we expect to re-discuss it occasionally, and we don’t find it to be more
urgent than features already on our full plate”.  Sometimes we file
tracking bugs for such features that are intentionally sidelined.

> - Profile/Archive score: I've been thinking that maybe a profile/archive score would be something useful to have at some point. The goal with such a "score" would be to report back to the user how much information from the archive was effectively used during a production run so that they have some insight about how "good" was the training run used to create the archive. The score for an archive could be an [weighted] aggregate of the scores of specific parts of the archive. We could also use the scores, in the future, if we choose to work on something to "merge" or "combine" multiple archives.

I mentally file ideas like this in “stuff the JVM should log about”.
There is a low barrier to entry for adding log items.  If such a bit of
information is useful for our end users we should provide a way to log it.

Some users will rapidly come to the place where the size (bulk, footprint)
of their AOT cache will push against limited resources (disk, bandwidth).
We need to be ready to help them “throttle back” the creation of AOT
resources.  This is one reason we periodically discuss ways to filter
the assets in the AOT cache.  We’ll need a story for this eventually,
though not immediately.  We also need a way, as you say, to evaluate
the result of a certain set of filter options.  For our own heuristics,
which determine “vanilla” behavior of the AOT cache, we also need the
similar evaluation of effectiveness.

One way to think about this is to give uniform names to AOT cache
assets, and use those names in two places in log output.  First,
when the asset is created (or when a hypothetical “dump command”
is executed), mention the asset.  (By asset I mean any AOT item:
klass, method, profile, nmethod, object, etc.)  Second, when the
asset is used, mention it again, and perhaps the circumstances of
its use.  A simple tool could then compare the list of created
assets against the list of used assets.  For klasses which are
loaded in bulk during premain, maybe that second report is not
interesting, but it might (again) be interesting to look at the
logged list of which classes were initialized, or maybe those
which were instantiated.  There’s lots of “in principle” work
here we could do, which might detour us into wasted effort,
so probably such logging work should be done in the setting
of problems we know users will eventually have, such as over-sized
AOT code caches.  That means we should work on code caching first,
then work on managing/curating/filtering/sizing the AOT code cache.

> - Condensers: I've been asked recently about the idea of condensers. As far as I understand there is nothing implemented on the lines of that idea. Is this something that we still plan to work in the future?

Given the “unreasonable effectiveness” of our current work, as Dan puts
it, we have placed such ideas low on the priority list.  Very low indeed.

Picking up on a theme Dan mentioned, I would like to work on bootstrapping
our indy/condy/MH/VMAC/LF initialization activities within premain, and not
as a separate tranche of engineering work.  This means making another look
at how java.lang.invoke APIs are wound up in Java application startup, and
finding ways to ensure that training runs “fill up the AOT cache” with useful
assets.  I don’t think anyone has looked at refactoring those initialization
activities, in the setting of premain, other than some work Vladimir Ivanov
did.  More work on indy/condy/MH/VMAC/LF initialization activities which
cooperate with premain (and especially pre-linking and cache filling
functions of premain) would bear more fruit, I think.

Such work might also reduce pressure on javac to provide special hacks
to avoid bootstrap problems in “the first 100 classes” of java.base.
IIRC we disable the indy-based string concat logic, when building
java.base, because there are too many bootstrap cycles.  I think a
premain-oriented approach to indy bootstrapping might possibly relieve
javac (and our makefiles) from pulling this trick.  The approach I
have in mind is to satisfy indy/condy/MH/VMAC/LF requests during
the earliest bootstrap phases (the “first 100”) by means of some
slow path that aligns well with the premain assembly phase.  That
is, arrange for the “first 1000” indy/condy/MH/VMAC/LF requests
to be fulfilled from a boot cache which is normally filled by
premain logic, and which can also be filled (in some unspecified
painful way) if the AOT cache is not present.  The unspecified
painful way might be a hand-curated set of pregenerated classes,
similar to work in the past on LFs.  These bootstrap-only paths
for handling that stuff would be code in java.base, not special
modes in javac, so they could be managed and minimized more directly.

(OTOH, some folks have said they like the simplicity of giving
javac a special output mode for java.base.  I’m not sure that will
scale given the many new indy-based language features arising.
As someone who works more on libraries than javac, my hammer
of choice is modification to library code, of course.)

> - Recompilation: I've been reviewing the changes in the @premain branch, and I came across this piece of code [1] and was puzzled by it. Why do we recompile the code at every compilation level? There is a lot I still don't understand about the implementation in premain, but what I was expecting was that at some point in the code we would just "dump" the code cache and that would be our "AOT compiled code".

That was our understanding at first.  It might seem that “all you need
to do” is snapshot the JVM’s JIT code cache at “the end of time” (after
training) and reuse it all at “the beginning of time” in the production run.

In reality, the premain training run emits (mostly) symbolic steering
information which causes the assembly phase to perform fresh compilations
of AOT methods.  And some of those AOT methods have to accommodate special
conditions at the “beginning of time” (such as not-yet-initialized classes).

If the application has a long self-configuration phase, the “end of time”
methods might actually deoptimize due to transient execution patterns
(eventually obsolete hot-spots); there might be a “middle of time” in
a large application which we need to cover somehow.  This is why there
is always an option to so online (JIT) compilation even when AOT methods
are available.  Also, if we have a very good “end of time” AOT method,
and it deoptimizes, the replacement policy should take into account
that there is evidence that it will eventually be useful.  So although
we tend to immediately dump JIT methods when they show the first sign
of unexpected behavior, AOT methods should be kept around longer.
And as such, they may have subtly different management of their own
internal fastpath/slowpath organization, compared to JIT methods.

Igor could tell you more…

> Thanks,
> Cesar
>
> [1] https://github.com/openjdk/leyden/blob/premain/src/hotspot/share/compiler/precompiler.cpp#L234