From Matthew.Carter at microsoft.com  Tue Nov  5 19:30:59 2024
From: Matthew.Carter at microsoft.com (Mat Carter)
Date: Tue, 5 Nov 2024 19:30:59 +0000
Subject: Optimal place for inserting calls to the VM from interpreted java
 methods
In-Reply-To: <56664e92-d012-48c6-8545-62cad29800d2@oracle.com>
References: <BL4PR21MB4128961B0C70498D8E4B8C9B8A782@BL4PR21MB4128.namprd21.prod.outlook.com>
 <56664e92-d012-48c6-8545-62cad29800d2@oracle.com>
Message-ID: <LV8PR21MB41455E4548B7A60899E703D28A522@LV8PR21MB4145.namprd21.prod.outlook.com>

Thanks Dean

That is indeed the way I ended up going.  I have a draft PR [1] up with these changes

Cheers
Mat

[1] https://github.com/openjdk/leyden/pull/21

From: leyden-dev <leyden-dev-retn at openjdk.org> on behalf of dean.long at oracle.com <dean.long at oracle.com>
Date: Friday, October 18, 2024 at 11:39 AM
To: leyden-dev at openjdk.org <leyden-dev at openjdk.org>
Subject: Re: Optimal place for inserting calls to the VM from interpreted java methods

Hi Mat.  It sounds like you could use multiple entry points based on MethodKind to accomplish this.  Add new entries to the MethodKind enum for the upcall versions.  Generate both the upcall and non-upcall entry points, and map to appropriate MethodKind values.  Then use your runtime pattern match to decide with MethodKind to use for each method in AbstractInterpreter<https://java.se.oracle.com/source/s?defs=AbstractInterpreter&project=jdk-cpu-git>::method_kind<https://java.se.oracle.com/source/s?defs=method_kind&project=jdk-cpu-git>(const methodHandle<https://java.se.oracle.com/source/s?defs=methodHandle&project=jdk-cpu-git>& m<https://java.se.oracle.com/source/s?defs=m&project=jdk-cpu-git>).

dl


On 10/9/24 6:42 PM, Mat Carter wrote:
Is there a place to encode calls to the VM in the interpreted methods other than the method prologs?
I've recently discovered the interpreted method adaptors which seems like a candidate, but there's
no examples of the adaptors calling into the VM.

Follows is the rational as to why I'm looking for an alternative to the method prologs

The AOTEndTrainingOnMethodEntry feature [1] introduces calls from java methods into the
VM (upcalls) when specific methods are entered. The methods are identified via a pattern in a
similar manner to the CompileOnly option.

Following the initial PR review we?re looking at removing the knowledge of this AOT feature from
the compilers/interpreter and introducing a more generic system (RuntimeUpcalls) that can be
used by other parts of the VM [2].  In building out the RuntimeUpcalls system we've come across
an inefficiency that isn't an immediate problem for this feature, but should another feature use this
new system then it's less than optimal.

Interpreted code uses a shared method prolog (there are 8 variants for 'regular' methods [more
for some special math/zip methods]), the AOTEndTrainingOnMethodEntry  feature introduces a
further 5 prolog types.  When there is a single upcall (eg. AOTEndTrainingOnMethodEntry) to the VM
everything is efficient.

The inefficiency issue arises as soon as there are two or more upcalls; which upcalls relate to
which methods is contained within the RuntimeUpcalls system.  When the interpreter examines
the method flags they only indicate whether there are any upcalls (but not how many or which ones).
As the interpreter can't encode which upcalls should be called in the prolog (without an explosion
of new runtime generated prologs), it needs to call the RuntimeUpcalls system which in turn iterates
over the upcalls and calls the appropriate ones; the problem is that during that iteration the methods
need to be compared against the pattern.  So we either pay a memory cost to cache the method to
upcall relationships or we pay a performance cost to repeatedly test the method against the pattern.

This is not a problem for C1 and C2 as we pay this cost only when the methods are compiled and
create the multiple upcalls in those methods, eliminating the need for pattern matching by the
RuntimeUpcalls system during method execution

Thanks in advance
Mat

[1] https://github.com/openjdk/leyden/pull/21
[2] https://github.com/macarte/leyden/pull/2
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/leyden-dev/attachments/20241105/ea1750bf/attachment.htm>

From kvn at openjdk.org  Wed Nov  6 22:51:15 2024
From: kvn at openjdk.org (Vladimir Kozlov)
Date: Wed, 6 Nov 2024 22:51:15 GMT
Subject: RFR: Save/load i2c2i adapters [v2]
In-Reply-To: <9bs4zYR_ctnBRwZu1yuQZvF-nTVOBzgmbDEnIwaUBhM=.e9f2ca96-4dcc-436e-9bf9-8456461ca46a@github.com>
References: <Xu6rsXHg2tAQvNlEXTR98POULaXj48G3YoiiLm4J9us=.913c6fa4-b569-4a51-999b-0fcbb3548a9c@github.com>
 <9bs4zYR_ctnBRwZu1yuQZvF-nTVOBzgmbDEnIwaUBhM=.e9f2ca96-4dcc-436e-9bf9-8456461ca46a@github.com>
Message-ID: <tAXc4JZrS1N8O7Yr2sDJYk1sKT65vw0_1QhX7VPXUoQ=.f6157503-603f-4e52-a2c7-8381c0db6876@github.com>

On Thu, 24 Oct 2024 15:59:03 GMT, Ashutosh Mehra <asmehra at openjdk.org> wrote:

>> This is an attempt to save and load i2c2i adapters along with the adapter handler table.
>> There are mainly two parts to this change:
>> 1. Storing of adapter code in the SCCache or AOT code cache.
>> 2. Storing of adapter handler table in the AOT cache.
>> 
>> Adapter handler table is a map from AdapterFingerPrint to AdapterHnadlerEntry. To store them in AOT cache, AdapterFingerPrint and AdapterHandlerEntry are updated to MetaspaceObj. Both these entities are discovered and added to the cache while processing the Method. When storing the adapter handler table, only the entries that have already been archived are considered. This allows pruning of AdapterHnadlerEntry that may be only reachable through a Method that is not eligible to be archived. 
>> 
>> An AdapterHandlerEntry has pointer to the adapter code. Because the AdapterHandlerEntry and the adapter code are stored in separate archives, this link between the AdapterHandlerEntry and the adapter code needs to be removed (see AdapterHandlerEntry::remove_unshareable_info()).
>> During the production run, as the methods in the AOT cache are adopted, the AdapterHandlerEntry is linked back to the adapter code (see AdapterHandlerEntry::restore_unshareable_info).
>> 
>> All this code is guarded by -XX:[+-]ArchiveAdapters option which defaults to false, but is set to true in CDSConfig during the assembly phase.
>> 
>> Other changes worth mentioning:
>> 1. Changes to the SCCache infrastructure to make it possible to store and load adapter code. (Thanks to @adinn)
>> 2. Updating AdapterFingerPrint hashing algorithm to avoid collisions. If there is any collision, then it will prevent finding the adapter code in the SCCache. (Again courtesy of @adinn)
>> 
>> Thanks to @adinn for providing many of these changes.
>> 
>> Performance:
>> -Xlog:init shows time taken for linking of Methods and making adapters. An example output is:
>> 
>> ClassLoader:
>>   clinit:                             150us / 4612 events
>>   link methods:                     28980us / 176893 events
>>   method adapters:                  15378us / 697 events
>> 
>> Save/load of adapters seem to have improved these stats.
>> 
>> | Quarkus | -ArchiveAdapters | +ArchiveAdapters |
>> |---|---|---|
>> | link methods | 12214us / 58913 events | 2700us / 58913 events |
>> | method adapters | 7793us / 607 events | 4402us / 38 events |
>> 
>> | Spring-petclinic | -ArchiveAdapters | +ArchiveAdapters |
>> |---|---|---|
>> | link methods | 28980us / 176893 events | 7485us / 176893 events |...
>
> Ashutosh Mehra has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Acquire Compile_lock when writing exception blobs to the SCCache
>   
>   Signed-off-by: Ashutosh Mehra <asmehra at redhat.com>

Very nice changes.  Few comments.

src/hotspot/share/code/SCCache.cpp line 595:

> 593:   SCCache* cache = SCCache::cache();
> 594:   if (cache != nullptr && cache->_table != nullptr) {
> 595:     cache->_table->init_extrs();

This is repetitive checks pattern. I suggest to replace it with check of call to:

  SCAddressTable* SCCache::addr_table() const {
    return is_on() && (cache()->_table != nullptr) ? cache()->_table : nullptr;
  }

src/hotspot/share/code/SCCache.cpp line 1175:

> 1173:           if (entries_address[i].comp_level() == CompLevel_none) {
> 1174:             shared_blobs_count++;
> 1175:           } else if (entries_address[i].comp_level() == CompLevel_simple) {

we also caching C1 code with simple profiling: `CompLevel_limited_profile`

src/hotspot/share/code/SCCache.cpp line 2488:

> 2486:   // TODO - maybe move this up to selected callers so we only lock
> 2487:   // when saving a c1 or opto blob
> 2488:   MutexLocker ml(Compile_lock);

Only one compiler thread should generate compiler's runtime blobs.
See `AbstractCompiler::should_perform_init()`.

src/hotspot/share/oops/method.cpp line 1271:

> 1269:   // TODO: how to identify code cache full situation now that the adapter() can be
> 1270:   // non-null if AOT cache is in use
> 1271: #if 0

Can you check next (opposite to check in the following assert)?:

 if (adapter() != nullptr && !adapter()->is_linked()) {

The assumption is that we have enough CodeCache when we loading adapters from APT cache. Otherwise we should bailout (did you test such case?).

Is `is_linked()` is specific for adapters from AOT cache?

-------------

PR Review: https://git.openjdk.org/leyden/pull/25#pullrequestreview-2419510084
PR Review Comment: https://git.openjdk.org/leyden/pull/25#discussion_r1831764076
PR Review Comment: https://git.openjdk.org/leyden/pull/25#discussion_r1831774205
PR Review Comment: https://git.openjdk.org/leyden/pull/25#discussion_r1831781219
PR Review Comment: https://git.openjdk.org/leyden/pull/25#discussion_r1831806413

From kvn at openjdk.org  Wed Nov  6 22:51:15 2024
From: kvn at openjdk.org (Vladimir Kozlov)
Date: Wed, 6 Nov 2024 22:51:15 GMT
Subject: RFR: Save/load i2c2i adapters [v2]
In-Reply-To: <tAXc4JZrS1N8O7Yr2sDJYk1sKT65vw0_1QhX7VPXUoQ=.f6157503-603f-4e52-a2c7-8381c0db6876@github.com>
References: <Xu6rsXHg2tAQvNlEXTR98POULaXj48G3YoiiLm4J9us=.913c6fa4-b569-4a51-999b-0fcbb3548a9c@github.com>
 <9bs4zYR_ctnBRwZu1yuQZvF-nTVOBzgmbDEnIwaUBhM=.e9f2ca96-4dcc-436e-9bf9-8456461ca46a@github.com>
 <tAXc4JZrS1N8O7Yr2sDJYk1sKT65vw0_1QhX7VPXUoQ=.f6157503-603f-4e52-a2c7-8381c0db6876@github.com>
Message-ID: <QNnf9nT2qNOsfPOjBqOBDYoDnLULUBRS2KCWeXRmFcg=.4d6651ca-9b88-42bf-ba2a-823a8045eb85@github.com>

On Wed, 6 Nov 2024 22:21:33 GMT, Vladimir Kozlov <kvn at openjdk.org> wrote:

>> Ashutosh Mehra has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Acquire Compile_lock when writing exception blobs to the SCCache
>>   
>>   Signed-off-by: Ashutosh Mehra <asmehra at redhat.com>
>
> src/hotspot/share/code/SCCache.cpp line 2488:
> 
>> 2486:   // TODO - maybe move this up to selected callers so we only lock
>> 2487:   // when saving a c1 or opto blob
>> 2488:   MutexLocker ml(Compile_lock);
> 
> Only one compiler thread should generate compiler's runtime blobs.
> See `AbstractCompiler::should_perform_init()`.

Unless you changed when you call this generation.

-------------

PR Review Comment: https://git.openjdk.org/leyden/pull/25#discussion_r1831782481

From iklam at openjdk.org  Thu Nov  7 17:15:14 2024
From: iklam at openjdk.org (Ioi Lam)
Date: Thu, 7 Nov 2024 17:15:14 GMT
Subject: RFR: Save/load i2c2i adapters [v2]
In-Reply-To: <9bs4zYR_ctnBRwZu1yuQZvF-nTVOBzgmbDEnIwaUBhM=.e9f2ca96-4dcc-436e-9bf9-8456461ca46a@github.com>
References: <Xu6rsXHg2tAQvNlEXTR98POULaXj48G3YoiiLm4J9us=.913c6fa4-b569-4a51-999b-0fcbb3548a9c@github.com>
 <9bs4zYR_ctnBRwZu1yuQZvF-nTVOBzgmbDEnIwaUBhM=.e9f2ca96-4dcc-436e-9bf9-8456461ca46a@github.com>
Message-ID: <L4U2yX9axt2osz-IPQl8wQXck-g5xqpR77ahyQp9YGw=.4ee0b050-21ed-4363-b67b-b3207d206704@github.com>

On Thu, 24 Oct 2024 15:59:03 GMT, Ashutosh Mehra <asmehra at openjdk.org> wrote:

>> This is an attempt to save and load i2c2i adapters along with the adapter handler table.
>> There are mainly two parts to this change:
>> 1. Storing of adapter code in the SCCache or AOT code cache.
>> 2. Storing of adapter handler table in the AOT cache.
>> 
>> Adapter handler table is a map from AdapterFingerPrint to AdapterHnadlerEntry. To store them in AOT cache, AdapterFingerPrint and AdapterHandlerEntry are updated to MetaspaceObj. Both these entities are discovered and added to the cache while processing the Method. When storing the adapter handler table, only the entries that have already been archived are considered. This allows pruning of AdapterHnadlerEntry that may be only reachable through a Method that is not eligible to be archived. 
>> 
>> An AdapterHandlerEntry has pointer to the adapter code. Because the AdapterHandlerEntry and the adapter code are stored in separate archives, this link between the AdapterHandlerEntry and the adapter code needs to be removed (see AdapterHandlerEntry::remove_unshareable_info()).
>> During the production run, as the methods in the AOT cache are adopted, the AdapterHandlerEntry is linked back to the adapter code (see AdapterHandlerEntry::restore_unshareable_info).
>> 
>> All this code is guarded by -XX:[+-]ArchiveAdapters option which defaults to false, but is set to true in CDSConfig during the assembly phase.
>> 
>> Other changes worth mentioning:
>> 1. Changes to the SCCache infrastructure to make it possible to store and load adapter code. (Thanks to @adinn)
>> 2. Updating AdapterFingerPrint hashing algorithm to avoid collisions. If there is any collision, then it will prevent finding the adapter code in the SCCache. (Again courtesy of @adinn)
>> 
>> Thanks to @adinn for providing many of these changes.
>> 
>> Performance:
>> -Xlog:init shows time taken for linking of Methods and making adapters. An example output is:
>> 
>> ClassLoader:
>>   clinit:                             150us / 4612 events
>>   link methods:                     28980us / 176893 events
>>   method adapters:                  15378us / 697 events
>> 
>> Save/load of adapters seem to have improved these stats.
>> 
>> | Quarkus | -ArchiveAdapters | +ArchiveAdapters |
>> |---|---|---|
>> | link methods | 12214us / 58913 events | 2700us / 58913 events |
>> | method adapters | 7793us / 607 events | 4402us / 38 events |
>> 
>> | Spring-petclinic | -ArchiveAdapters | +ArchiveAdapters |
>> |---|---|---|
>> | link methods | 28980us / 176893 events | 7485us / 176893 events |...
>
> Ashutosh Mehra has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Acquire Compile_lock when writing exception blobs to the SCCache
>   
>   Signed-off-by: Ashutosh Mehra <asmehra at redhat.com>

src/hotspot/share/oops/method.cpp line 432:

> 430:   if (_adapter != nullptr) {
> 431:     _adapter->restore_unshareable_info(CHECK);
> 432:     _from_compiled_entry = _adapter->get_c2i_entry();

There's similar code in `Method::restore_archived_method_handle_intrinsic()`. I would suggest combining them into a helper function to avoid code duplication.

-------------

PR Review Comment: https://git.openjdk.org/leyden/pull/25#discussion_r1833055300

From ioi.lam at oracle.com  Thu Nov  7 20:48:47 2024
From: ioi.lam at oracle.com (ioi.lam at oracle.com)
Date: Thu, 7 Nov 2024 12:48:47 -0800
Subject: Skeleton code for mapping code cache from CDS
Message-ID: <0a8502ee-55db-463d-b844-95d655118c7b@oracle.com>

In follow up to today's Leyden meeting:

Here's the skeleton code in the premain branch that allows you to

- allocate space from the "cached code" section of the CDS archive

- Store pointers to (a) metadata and (b) other objects that are 
allocated within "cached code" section. These pointers will be 
automatically relocated during the production run.

https://github.com/openjdk/leyden/blob/21640b10188c8bfb414505043110d6da15080d16/src/hotspot/share/code/SCCache.cpp#L429-L504


(Note: currently, the AOT-compiled methods are NOT stored in this 
section, but are stored in a separate file).


From john.r.rose at oracle.com  Thu Nov  7 21:58:34 2024
From: john.r.rose at oracle.com (John Rose)
Date: Thu, 07 Nov 2024 13:58:34 -0800
Subject: cleanups to do after JEP 483 push
Message-ID: <33821D29-C060-4F9F-ADFC-60455FD14308@oracle.com>

Now that our ?JEP One? (483) is nearing the target stage,
it is time to talk about post-push work.  That would be small
stuff like bug fixes, refactorings (for tech debt payoff),
and API adjustments. The big stuff (JEPs Two and Three for
persistent profiles and AOT code) will follow in time.

Here?s my current list of possible post-483 cleanups.
We don?t need to do them all, nor necessarily in JDK 24.

- contract adjustments for stable, and usage corrections
- Mat C.?s hook for ending training runs (small API upgrade)
- fix for handling signed JARs (minor bug in PIT)
- fix for class loader table overflow (minor bug in PIT)
- cleanups for bootstrap orchestration (this is open-ended)
- clarify user model for integer box cache, when using AOT cache
- get rid of the ?scratch object? lifecycle in AOT processing

Let?s track this ?wish list? under [JDK-8343023], please.
And start proper RFEs or bugs as each item gets momentum.

And, do reply with more suggestions.  Bigger, longer range
items are welcome also, but let?s keep them distinct in our
minds.  I?m mainly talking here about stuff that will make
JEP One more robust and maintainable, and a better foundation
for JEPs Two, Three, and so on.

More to follow?

? John

[JDK-8343023]: https://bugs.openjdk.org/browse/JDK-8343023
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/leyden-dev/attachments/20241107/a9ff60ef/attachment.htm>

From john.r.rose at oracle.com  Thu Nov  7 21:58:48 2024
From: john.r.rose at oracle.com (John Rose)
Date: Thu, 07 Nov 2024 13:58:48 -0800
Subject: orchestration of early boot phases of JVM/JDK
Message-ID: <733D7D3E-1563-4539-95DF-F1D963C53A1C@oracle.com>


One bit of Leyden cleanup I want to tackle is regularizing
the early phases of JVM/JDK initialization, where the JVM
C++ code loads a small handful of classes, and then calls
a few Java methods, as it also warms up its C++ data
structures (such as JIT queues).

The Java methods are (increasingly) centralized in the
class jl.System.  I think this is a good move; it makes
the special conditions of early bootstrap easier to
understand, by inspecting the Java code.

In the same direction, I am proposing some cleanups to
jl.System and related API points:

https://github.com/openjdk/jdk/pull/21514

Over time, I would like to see LESS C++ code and MORE
Java code responsible for orchestrating the early boot
activities.

Another place where there is (IMO) more C++ code than
we want is in the C++ lists of pre-loaded JDK classes.
I would like to see these lists get shorter, or even
disappear.  Ideally it should be enough (say) to initialize
jl.System and call its ?set-init-level? method several
times, from C++, and let the JVM and JDK code do the rest.

I would also like to explore marking JDK classes and
methods with annotations (JDK-private, like Stable)
when those classes and methods have special roles in
the Leyden life cycle, or even the bare Leyden-free
?exploded build? life cycle.  When we pack up the
MethodType table at the end of the Assembly Phase,
or clear out overflowing tables (at the same time),
or when we unpack something in a runtime-startup
method, or when we query the number of processors
(or some other environmental thingy) at the right
moment, if the bootstrap sequence differs between
a bare Leyden-free run, and a run with a big AOT
cache, the difference should be really clearly
marked in the JDK code, not buried in the C++ code.

We are handling all these issues in an ad hoc way
in Ioi?s current JEP One PR.  We should handle them
in a less ad hoc way, slowly putting the relevant
configuration information in the right places.
Eventually (not immediately) we will understand
how to build a user model that is clean and sane
enough for third parties to use as well.

? John

From duke at openjdk.org  Thu Nov  7 23:54:56 2024
From: duke at openjdk.org (duke)
Date: Thu, 7 Nov 2024 23:54:56 GMT
Subject: git: openjdk/leyden: premain: fix -Xlog:init output
Message-ID: <e089c2ae-fa48-4bf5-9e4f-7acabf00e35b@openjdk.org>

Changeset: 52eee916
Branch: premain
Author:    Vladimir Ivanov <vladimir.x.ivanov at oracle.com>
Date:      2024-11-07 15:53:09 +0000
URL:       https://git.openjdk.org/leyden/commit/52eee916777c6b7a7a70ea6f55750935e0033d87

fix -Xlog:init output

! src/hotspot/share/cds/aotLinkedClassBulkLoader.cpp
! src/hotspot/share/cds/aotLinkedClassBulkLoader.hpp
! src/hotspot/share/classfile/classLoader.cpp
! src/hotspot/share/runtime/java.cpp
! src/hotspot/share/runtime/java.hpp
! src/hotspot/share/runtime/sharedRuntime.cpp
! src/hotspot/share/runtime/threads.cpp


From duke at openjdk.org  Mon Nov 11 22:17:51 2024
From: duke at openjdk.org (duke)
Date: Mon, 11 Nov 2024 22:17:51 GMT
Subject: git: openjdk/leyden: hermetic-java-runtime: Make error() a static
 function in childproc.c. This is to resolve `ld: error: duplicate symbol:
 error` issue on hermetic Java with libjava.a and third party library.
Message-ID: <e4d2c234-a9a9-425a-b7c3-dd26dbfcd4de@openjdk.org>

Changeset: e20a9155
Branch: hermetic-java-runtime
Author:    Jiangli Zhou <jianglizhou at google.com>
Date:      2024-11-11 14:14:56 +0000
URL:       https://git.openjdk.org/leyden/commit/e20a9155ae37caa5151005997fc9b4478bb2038d

Make error() a static function in childproc.c. This is to resolve `ld: error: duplicate symbol: error` issue on hermetic Java with libjava.a and third party library.

! src/java.base/unix/native/libjava/childproc.c


From duke at openjdk.org  Tue Nov 12 07:19:00 2024
From: duke at openjdk.org (duke)
Date: Tue, 12 Nov 2024 07:19:00 GMT
Subject: git: openjdk/leyden: created branch
 8341587-example-of-weakref-problem-in-aot-cache based on the branch premain
 containing 1 unique commit
Message-ID: <4936881f-b2ca-4d20-b73a-ad0901a45cdd@openjdk.org>

The following commits are unique to the 8341587-example-of-weakref-problem-in-aot-cache branch:
========================================================
4730652e: An example of WeakReference not working when stored inside the AOT cache

From asmehra at openjdk.org  Tue Nov 12 18:38:11 2024
From: asmehra at openjdk.org (Ashutosh Mehra)
Date: Tue, 12 Nov 2024 18:38:11 GMT
Subject: RFR: Save/load i2c2i adapters [v3]
In-Reply-To: <Xu6rsXHg2tAQvNlEXTR98POULaXj48G3YoiiLm4J9us=.913c6fa4-b569-4a51-999b-0fcbb3548a9c@github.com>
References: <Xu6rsXHg2tAQvNlEXTR98POULaXj48G3YoiiLm4J9us=.913c6fa4-b569-4a51-999b-0fcbb3548a9c@github.com>
Message-ID: <MyJ_GBtEeYA4oeetuFWzgxe_CIASMUNojLdwNRJPC-E=.8d8d1c9d-a09f-4ddd-8450-d4d5bc8f78f6@github.com>

> This is an attempt to save and load i2c2i adapters along with the adapter handler table.
> There are mainly two parts to this change:
> 1. Storing of adapter code in the SCCache or AOT code cache.
> 2. Storing of adapter handler table in the AOT cache.
> 
> Adapter handler table is a map from AdapterFingerPrint to AdapterHnadlerEntry. To store them in AOT cache, AdapterFingerPrint and AdapterHandlerEntry are updated to MetaspaceObj. Both these entities are discovered and added to the cache while processing the Method. When storing the adapter handler table, only the entries that have already been archived are considered. This allows pruning of AdapterHnadlerEntry that may be only reachable through a Method that is not eligible to be archived. 
> 
> An AdapterHandlerEntry has pointer to the adapter code. Because the AdapterHandlerEntry and the adapter code are stored in separate archives, this link between the AdapterHandlerEntry and the adapter code needs to be removed (see AdapterHandlerEntry::remove_unshareable_info()).
> During the production run, as the methods in the AOT cache are adopted, the AdapterHandlerEntry is linked back to the adapter code (see AdapterHandlerEntry::restore_unshareable_info).
> 
> All this code is guarded by -XX:[+-]ArchiveAdapters option which defaults to false, but is set to true in CDSConfig during the assembly phase.
> 
> Other changes worth mentioning:
> 1. Changes to the SCCache infrastructure to make it possible to store and load adapter code. (Thanks to @adinn)
> 2. Updating AdapterFingerPrint hashing algorithm to avoid collisions. If there is any collision, then it will prevent finding the adapter code in the SCCache. (Again courtesy of @adinn)
> 
> Thanks to @adinn for providing many of these changes.
> 
> Performance:
> -Xlog:init shows time taken for linking of Methods and making adapters. An example output is:
> 
> ClassLoader:
>   clinit:                             150us / 4612 events
>   link methods:                     28980us / 176893 events
>   method adapters:                  15378us / 697 events
> 
> Save/load of adapters seem to have improved these stats.
> 
> | Quarkus | -ArchiveAdapters | +ArchiveAdapters |
> |---|---|---|
> | link methods | 12214us / 58913 events | 2700us / 58913 events |
> | method adapters | 7793us / 607 events | 4402us / 38 events |
> 
> | Spring-petclinic | -ArchiveAdapters | +ArchiveAdapters |
> |---|---|---|
> | link methods | 28980us / 176893 events | 7485us / 176893 events |
> | method adapters | 15378us / 697 events | 7050us / 13 events |
> 
> Howeve...

Ashutosh Mehra has updated the pull request incrementally with two additional commits since the last revision:

 - Address review comments by iklam
   
   Signed-off-by: Ashutosh Mehra <asmehra at redhat.com>
 - Address reivew comments by vnkozlov
   
   Signed-off-by: Ashutosh Mehra <asmehra at redhat.com>

-------------

Changes:
  - all: https://git.openjdk.org/leyden/pull/25/files
  - new: https://git.openjdk.org/leyden/pull/25/files/b04ba154..59b65ef1

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=leyden&pr=25&range=02
 - incr: https://webrevs.openjdk.org/?repo=leyden&pr=25&range=01-02

  Stats: 38 lines in 4 files changed: 12 ins; 6 del; 20 mod
  Patch: https://git.openjdk.org/leyden/pull/25.diff
  Fetch: git fetch https://git.openjdk.org/leyden.git pull/25/head:pull/25

PR: https://git.openjdk.org/leyden/pull/25

From asmehra at openjdk.org  Tue Nov 12 18:38:11 2024
From: asmehra at openjdk.org (Ashutosh Mehra)
Date: Tue, 12 Nov 2024 18:38:11 GMT
Subject: RFR: Save/load i2c2i adapters [v2]
In-Reply-To: <QNnf9nT2qNOsfPOjBqOBDYoDnLULUBRS2KCWeXRmFcg=.4d6651ca-9b88-42bf-ba2a-823a8045eb85@github.com>
References: <Xu6rsXHg2tAQvNlEXTR98POULaXj48G3YoiiLm4J9us=.913c6fa4-b569-4a51-999b-0fcbb3548a9c@github.com>
 <9bs4zYR_ctnBRwZu1yuQZvF-nTVOBzgmbDEnIwaUBhM=.e9f2ca96-4dcc-436e-9bf9-8456461ca46a@github.com>
 <tAXc4JZrS1N8O7Yr2sDJYk1sKT65vw0_1QhX7VPXUoQ=.f6157503-603f-4e52-a2c7-8381c0db6876@github.com>
 <QNnf9nT2qNOsfPOjBqOBDYoDnLULUBRS2KCWeXRmFcg=.4d6651ca-9b88-42bf-ba2a-823a8045eb85@github.com>
Message-ID: <HPvRZj51AnJ8VbWHaa1gspexruQeCwKnT6k18szxO9Y=.e60d2fba-f298-4b98-a6b0-4767e31a8e37@github.com>

On Wed, 6 Nov 2024 22:23:07 GMT, Vladimir Kozlov <kvn at openjdk.org> wrote:

>> src/hotspot/share/code/SCCache.cpp line 2488:
>> 
>>> 2486:   // TODO - maybe move this up to selected callers so we only lock
>>> 2487:   // when saving a c1 or opto blob
>>> 2488:   MutexLocker ml(Compile_lock);
>> 
>> Only one compiler thread should generate compiler's runtime blobs.
>> See `AbstractCompiler::should_perform_init()`.
>
> Unless you changed when you call this generation.

Done

-------------

PR Review Comment: https://git.openjdk.org/leyden/pull/25#discussion_r1838588469

From asmehra at openjdk.org  Tue Nov 12 18:38:11 2024
From: asmehra at openjdk.org (Ashutosh Mehra)
Date: Tue, 12 Nov 2024 18:38:11 GMT
Subject: RFR: Save/load i2c2i adapters [v2]
In-Reply-To: <tAXc4JZrS1N8O7Yr2sDJYk1sKT65vw0_1QhX7VPXUoQ=.f6157503-603f-4e52-a2c7-8381c0db6876@github.com>
References: <Xu6rsXHg2tAQvNlEXTR98POULaXj48G3YoiiLm4J9us=.913c6fa4-b569-4a51-999b-0fcbb3548a9c@github.com>
 <9bs4zYR_ctnBRwZu1yuQZvF-nTVOBzgmbDEnIwaUBhM=.e9f2ca96-4dcc-436e-9bf9-8456461ca46a@github.com>
 <tAXc4JZrS1N8O7Yr2sDJYk1sKT65vw0_1QhX7VPXUoQ=.f6157503-603f-4e52-a2c7-8381c0db6876@github.com>
Message-ID: <fxJGsZQEKYv0_47XxBbglmenqLgYbpZrirquY3vuK_4=.9ffbb9f7-a778-4f4e-8a08-4a59985265ee@github.com>

On Wed, 6 Nov 2024 22:01:53 GMT, Vladimir Kozlov <kvn at openjdk.org> wrote:

>> Ashutosh Mehra has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Acquire Compile_lock when writing exception blobs to the SCCache
>>   
>>   Signed-off-by: Ashutosh Mehra <asmehra at redhat.com>
>
> src/hotspot/share/code/SCCache.cpp line 595:
> 
>> 593:   SCCache* cache = SCCache::cache();
>> 594:   if (cache != nullptr && cache->_table != nullptr) {
>> 595:     cache->_table->init_extrs();
> 
> This is repetitive checks pattern. I suggest to replace it with check of call to:
> 
>   SCAddressTable* SCCache::addr_table() const {
>     return is_on() && (cache()->_table != nullptr) ? cache()->_table : nullptr;
>   }

Done

> src/hotspot/share/code/SCCache.cpp line 1175:
> 
>> 1173:           if (entries_address[i].comp_level() == CompLevel_none) {
>> 1174:             shared_blobs_count++;
>> 1175:           } else if (entries_address[i].comp_level() == CompLevel_simple) {
> 
> we also caching C1 code with simple profiling: `CompLevel_limited_profile`

Added check for `CompLevel_limited_profile`

-------------

PR Review Comment: https://git.openjdk.org/leyden/pull/25#discussion_r1838588288
PR Review Comment: https://git.openjdk.org/leyden/pull/25#discussion_r1838589064

From asmehra at openjdk.org  Tue Nov 12 18:38:12 2024
From: asmehra at openjdk.org (Ashutosh Mehra)
Date: Tue, 12 Nov 2024 18:38:12 GMT
Subject: RFR: Save/load i2c2i adapters [v2]
In-Reply-To: <L4U2yX9axt2osz-IPQl8wQXck-g5xqpR77ahyQp9YGw=.4ee0b050-21ed-4363-b67b-b3207d206704@github.com>
References: <Xu6rsXHg2tAQvNlEXTR98POULaXj48G3YoiiLm4J9us=.913c6fa4-b569-4a51-999b-0fcbb3548a9c@github.com>
 <9bs4zYR_ctnBRwZu1yuQZvF-nTVOBzgmbDEnIwaUBhM=.e9f2ca96-4dcc-436e-9bf9-8456461ca46a@github.com>
 <L4U2yX9axt2osz-IPQl8wQXck-g5xqpR77ahyQp9YGw=.4ee0b050-21ed-4363-b67b-b3207d206704@github.com>
Message-ID: <Q_TDfIsq7XNstALffR41gqUBvjJoW6CBc-nV8mMtWyc=.bac6a42c-7545-4fa4-b717-ef33b0f7fc33@github.com>

On Thu, 7 Nov 2024 17:11:26 GMT, Ioi Lam <iklam at openjdk.org> wrote:

>> Ashutosh Mehra has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Acquire Compile_lock when writing exception blobs to the SCCache
>>   
>>   Signed-off-by: Ashutosh Mehra <asmehra at redhat.com>
>
> src/hotspot/share/oops/method.cpp line 432:
> 
>> 430:   if (_adapter != nullptr) {
>> 431:     _adapter->restore_unshareable_info(CHECK);
>> 432:     _from_compiled_entry = _adapter->get_c2i_entry();
> 
> There's similar code in `Method::restore_archived_method_handle_intrinsic()`. I would suggest combining them into a helper function to avoid code duplication.

Done

-------------

PR Review Comment: https://git.openjdk.org/leyden/pull/25#discussion_r1838588662

From asmehra at openjdk.org  Tue Nov 12 19:19:43 2024
From: asmehra at openjdk.org (Ashutosh Mehra)
Date: Tue, 12 Nov 2024 19:19:43 GMT
Subject: RFR: Save/load i2c2i adapters [v2]
In-Reply-To: <tAXc4JZrS1N8O7Yr2sDJYk1sKT65vw0_1QhX7VPXUoQ=.f6157503-603f-4e52-a2c7-8381c0db6876@github.com>
References: <Xu6rsXHg2tAQvNlEXTR98POULaXj48G3YoiiLm4J9us=.913c6fa4-b569-4a51-999b-0fcbb3548a9c@github.com>
 <9bs4zYR_ctnBRwZu1yuQZvF-nTVOBzgmbDEnIwaUBhM=.e9f2ca96-4dcc-436e-9bf9-8456461ca46a@github.com>
 <tAXc4JZrS1N8O7Yr2sDJYk1sKT65vw0_1QhX7VPXUoQ=.f6157503-603f-4e52-a2c7-8381c0db6876@github.com>
Message-ID: <1VLjT_Dhcuc0oxW3kzlxxybaaJPBDO0fvIaEAFxxGNc=.e8d3f1bb-474f-42bf-b2f0-3d1a736c0ef4@github.com>

On Wed, 6 Nov 2024 22:47:29 GMT, Vladimir Kozlov <kvn at openjdk.org> wrote:

>> Ashutosh Mehra has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Acquire Compile_lock when writing exception blobs to the SCCache
>>   
>>   Signed-off-by: Ashutosh Mehra <asmehra at redhat.com>
>
> src/hotspot/share/oops/method.cpp line 1271:
> 
>> 1269:   // TODO: how to identify code cache full situation now that the adapter() can be
>> 1270:   // non-null if AOT cache is in use
>> 1271: #if 0
> 
> Can you check next (opposite to check in the following assert)?:
> 
>  if (adapter() != nullptr && !adapter()->is_linked()) {
> 
> The assumption is that we have enough CodeCache when we loading adapters from APT cache. Otherwise we should bailout (did you test such case?).
> 
> Is `is_linked()` is specific for adapters from AOT cache?

`is_linked()` is being set for every AdapterHandlerEntry when its code is either generated or loaded from AOT cache.

Regarding the original block of code that this check pertains to:


  // If the code cache is full, we may reenter this function for the
  // leftover methods that weren't linked.
  if (adapter() != nullptr) {
    return;
  }


The comment seem to indicate that we may reenter this function for a Method* for which adapter code has already been generated. However I am not able to trace the code path that may result in re-entering this function. Can you please explain under what conditions is this possible? @vnkozlov

-------------

PR Review Comment: https://git.openjdk.org/leyden/pull/25#discussion_r1838637737

From duke at openjdk.org  Wed Nov 13 03:35:18 2024
From: duke at openjdk.org (duke)
Date: Wed, 13 Nov 2024 03:35:18 GMT
Subject: git: openjdk/leyden: premain: Cleanup includes
Message-ID: <e92bf382-6f47-4832-bbef-5d31af4d84c0@openjdk.org>

Changeset: 1c6e576c
Branch: premain
Author:    Igor Veresov <igor.veresov at oracle.com>
Date:      2024-11-12 19:34:02 +0000
URL:       https://git.openjdk.org/leyden/commit/1c6e576cb55c2da118afdfda6a450259bfe4e42c

Cleanup includes

! src/hotspot/share/cds/aotClassInitializer.cpp
! src/hotspot/share/cds/methodDataDictionary.hpp
! src/hotspot/share/compiler/precompiler.cpp
! src/hotspot/share/oops/recompilationSchedule.cpp
! src/hotspot/share/oops/trainingData.cpp
! src/hotspot/share/oops/trainingData.hpp


From duke at openjdk.org  Wed Nov 13 23:34:49 2024
From: duke at openjdk.org (duke)
Date: Wed, 13 Nov 2024 23:34:49 GMT
Subject: git: openjdk/leyden: premain: Remove MethodInfoDictionary
Message-ID: <336bac9d-d9c8-42cd-bb14-62987ed93f4f@openjdk.org>

Changeset: cd8c4f40
Branch: premain
Author:    Igor Veresov <igor.veresov at oracle.com>
Date:      2024-11-13 15:32:38 +0000
URL:       https://git.openjdk.org/leyden/commit/cd8c4f4002eb435d8ea8db5c70e9b9b1de7e46eb

Remove MethodInfoDictionary

! src/hotspot/share/cds/archiveBuilder.cpp
! src/hotspot/share/cds/dynamicArchive.cpp
! src/hotspot/share/cds/metaspaceShared.cpp
- src/hotspot/share/cds/methodDataDictionary.cpp
- src/hotspot/share/cds/methodDataDictionary.hpp
! src/hotspot/share/classfile/systemDictionaryShared.cpp
! src/hotspot/share/classfile/systemDictionaryShared.hpp
! src/hotspot/share/oops/trainingData.hpp


From macarte at openjdk.org  Thu Nov 14 20:01:36 2024
From: macarte at openjdk.org (Mat Carter)
Date: Thu, 14 Nov 2024 20:01:36 GMT
Subject: RFR: 8335358: [premain] Explore alternative ways to trigger the
 end of training run [v11]
In-Reply-To: <RqVNoxdgqloUtbXxgiiqePA6XRrHkuolETkbzmzr7r4=.2b32af0e-69fa-48cf-89aa-322371ef9fb2@github.com>
References: <RqVNoxdgqloUtbXxgiiqePA6XRrHkuolETkbzmzr7r4=.2b32af0e-69fa-48cf-89aa-322371ef9fb2@github.com>
Message-ID: <eFkKB_LXKrDNgSj-EV7xXDo7JqNdhNNM93EVvI5VaT0=.4c7cd29e-f4e3-4430-9fda-f9fa784093e8@github.com>

> AOT training can be ended using either
> 
> - -XX: AOTEndTrainingOnMethodEntry =Hello.someMethod [same syntax as CompileOnly]
> - -XX: AOTEndTrainingOnMethodEntry =Hello.someMethod,Hello.someOtherMethod,count=42
> - jcmd <pid> AOT.end_training
> 
> supports arm64 and x64
> 
> note: the AOTEndTrainingOnMethodEntry is ignored when not recording
> 
> JBS Issue: https://bugs.openjdk.org/browse/JDK-8335358

Mat Carter has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 43 additional commits since the last revision:

 - Enhanced upcall system testing options (based on PR feedback)
 - Removing unused includes
 - Remove cds include, no longer required after refactor
 - merge issue
 - fix merge issues
 - Update other platforms
 - Merge branch 'premain' into macarte-endtraining
 - Merge pull request #2 from macarte/macarte-endtraining-refactor
   
   Macarte endtraining refactor
 - minor edits
 - small changes based on PR review
 - ... and 33 more: https://git.openjdk.org/leyden/compare/4af49dbe...9c3cdf78

-------------

Changes:
  - all: https://git.openjdk.org/leyden/pull/21/files
  - new: https://git.openjdk.org/leyden/pull/21/files/63d60ae9..9c3cdf78

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=leyden&pr=21&range=10
 - incr: https://webrevs.openjdk.org/?repo=leyden&pr=21&range=09-10

  Stats: 238173 lines in 2437 files changed: 209373 ins; 15575 del; 13225 mod
  Patch: https://git.openjdk.org/leyden/pull/21.diff
  Fetch: git fetch https://git.openjdk.org/leyden.git pull/21/head:pull/21

PR: https://git.openjdk.org/leyden/pull/21

From macarte at openjdk.org  Thu Nov 14 20:01:37 2024
From: macarte at openjdk.org (Mat Carter)
Date: Thu, 14 Nov 2024 20:01:37 GMT
Subject: RFR: 8335358: [premain] Explore alternative ways to trigger the
 end of training run [v9]
In-Reply-To: <d6t_EWfaQsUBe4qMFAQiKZmHZjtfhyTKH0LUCiVL1-Y=.f9a80e0f-8525-4c1c-84e8-bb80237fe07b@github.com>
References: <RqVNoxdgqloUtbXxgiiqePA6XRrHkuolETkbzmzr7r4=.2b32af0e-69fa-48cf-89aa-322371ef9fb2@github.com>
 <u-mV8QEX6TlfZmz-t-yeFp7CJNjAQzyN9Qr_10NEu90=.778034a0-d6eb-44a8-8c8c-67a75cccff18@github.com>
 <d6t_EWfaQsUBe4qMFAQiKZmHZjtfhyTKH0LUCiVL1-Y=.f9a80e0f-8525-4c1c-84e8-bb80237fe07b@github.com>
Message-ID: <RZgymouejTif_mOAUQsvUGoRUPsk54W_xr89HYub1FY=.fdab65c5-d288-475d-9849-7b51ce51daed@github.com>

On Thu, 19 Sep 2024 05:42:05 GMT, John R Rose <jrose at openjdk.org> wrote:

> Suggestion: Rename end_training_check in the interpreter to method_entry_upcall. That way, the very wide but shallow changes to the assembly code will support a variety of features, not just this particular one.
> 
> Probably the same thing would work for the JITs as well. So when C1 says
> 
> ```
> CDSConfig::is_dumping_preimage_static_archive_with_triggers() && callee->is_end_training_trigger()
> ```
> 
> Basically, the assembler and interpreter and JITs should not be thinking about CDS, just this newfangled "upcall".
> 
> it might as well say `callee->has_entry_upcall()` and the CDS-specific logic can be moved nearer to the method classes. The name `is_end_training_trigger` can be retired or used in fewer places, when the only question at hand is whether to stick in an upcall. Perhaps methodFlags continues to have `is_end_training_trigger` but `Method` exports `has_upcall` which delegates (internally) to `is_end_training_trigger`.
> 
> When you get to the C++ code for the upcall, just proceed as you do already, with a comment that the upcall has been installed if and only if the method was marked for training end. Later on, that logic (again in C++) can be extended if we have other entry methods we wish to track. Or not; maybe we'll never have a second use for this mechanism. But even then, a more general looking name won't be harmful. And there is some value in keeping the CDS-specific logic more localized, in a handful of functions that handle "upcalls" whatever those might be.
> 
> There are other lightweight versions of upcalls floating around too, low-level method entry notifications. Some of them have been prototyped for Leyden (to trigger slow-burning recompilations from optimized code). Factoring this feature into an upcall pattern might make it easier to merge the various kinds of upcalls into a common facility, eventually.
> 
> The name `_end_training_predicate` promises more than it is; shouldn't it be `_end_training_count_limit`?
> 
> I suggest passing the `CompileCommandEnum` into `parse_method_pattern`. This will be a more user-facing instance of method patterns, so there will probably be special rules written for them. Probably more restrictive than the present rules for method patterns in other compile commands. For starters, we should choose between the dot-based and double-colon-based syntaxes. I can't predict which way this conversation will go, but it will happen and we will be glad to be able to contextualize the parse to this particular command.

Thanks again for the feedback, I've updated this (now draft) PR with these comments in mind, please take a look - will leave as draft PR until discussed

-------------

PR Comment: https://git.openjdk.org/leyden/pull/21#issuecomment-2458002003

From asmehra at openjdk.org  Thu Nov 14 20:01:38 2024
From: asmehra at openjdk.org (Ashutosh Mehra)
Date: Thu, 14 Nov 2024 20:01:38 GMT
Subject: RFR: 8335358: [premain] Explore alternative ways to trigger the
 end of training run [v9]
In-Reply-To: <RZgymouejTif_mOAUQsvUGoRUPsk54W_xr89HYub1FY=.fdab65c5-d288-475d-9849-7b51ce51daed@github.com>
References: <RqVNoxdgqloUtbXxgiiqePA6XRrHkuolETkbzmzr7r4=.2b32af0e-69fa-48cf-89aa-322371ef9fb2@github.com>
 <u-mV8QEX6TlfZmz-t-yeFp7CJNjAQzyN9Qr_10NEu90=.778034a0-d6eb-44a8-8c8c-67a75cccff18@github.com>
 <d6t_EWfaQsUBe4qMFAQiKZmHZjtfhyTKH0LUCiVL1-Y=.f9a80e0f-8525-4c1c-84e8-bb80237fe07b@github.com>
 <RZgymouejTif_mOAUQsvUGoRUPsk54W_xr89HYub1FY=.fdab65c5-d288-475d-9849-7b51ce51daed@github.com>
Message-ID: <V9Z5Z73OzA5Drcng7Bwwz06vVZXlAiv8W8D-GjDzsQg=.baef1a5f-1d9d-4860-bbb3-cc27d4554a5a@github.com>

On Tue, 5 Nov 2024 19:33:36 GMT, Mat Carter <macarte at openjdk.org> wrote:

>> Suggestion:  Rename end_training_check in the interpreter to method_entry_upcall.  That way, the very wide but shallow changes to the assembly code will support a variety of features, not just this particular one.
>> 
>> Probably the same thing would work for the JITs as well.  So when C1 says
>> 
>> 
>> CDSConfig::is_dumping_preimage_static_archive_with_triggers() && callee->is_end_training_trigger()
>> 
>> 
>> Basically, the assembler and interpreter and JITs should not be thinking about CDS, just this newfangled "upcall".
>> 
>> it might as well say `callee->has_entry_upcall()` and the CDS-specific logic can be moved nearer to the method classes.  The name `is_end_training_trigger` can be retired or used in fewer places, when the only question at hand is whether to stick in an upcall.  Perhaps methodFlags continues to have `is_end_training_trigger` but `Method` exports `has_upcall` which delegates (internally) to `is_end_training_trigger`.
>> 
>> When you get to the C++ code for the upcall, just proceed as you do already, with a comment that the upcall has been installed if and only if the method was marked for training end.  Later on, that logic (again in C++) can be extended if we have other entry methods we wish to track.  Or not; maybe we'll never have a second use for this mechanism.  But even then, a more general looking name won't be harmful.  And there is some value in keeping the CDS-specific logic more localized, in a handful of functions that handle "upcalls" whatever those might be.
>> 
>> There are other lightweight versions of upcalls floating around too, low-level method entry notifications.  Some of them have been prototyped for Leyden (to trigger slow-burning recompilations from optimized code).  Factoring this feature into an upcall pattern might make it easier to merge the various kinds of upcalls into a common facility, eventually.
>> 
>> The name `_end_training_predicate` promises more than it is; shouldn't it be `_end_training_count_limit`?
>> 
>> I suggest passing the `CompileCommandEnum` into `parse_method_pattern`.  This will be a more user-facing instance of method patterns, so there will probably be special rules written for them.  Probably more restrictive than the present rules for method patterns in other compile commands.  For starters, we should choose between the dot-based and double-colon-based syntaxes.  I can't predict which way this conversation will go, but it will happen and we will be glad to be able to contextualize the parse to...
>
>> Suggestion: Rename end_training_check in the interpreter to method_entry_upcall. That way, the very wide but shallow changes to the assembly code will support a variety of features, not just this particular one.
>> 
>> Probably the same thing would work for the JITs as well. So when C1 says
>> 
>> ```
>> CDSConfig::is_dumping_preimage_static_archive_with_triggers() && callee->is_end_training_trigger()
>> ```
>> 
>> Basically, the assembler and interpreter and JITs should not be thinking about CDS, just this newfangled "upcall".
>> 
>> it might as well say `callee->has_entry_upcall()` and the CDS-specific logic can be moved nearer to the method classes. The name `is_end_training_trigger` can be retired or used in fewer places, when the only question at hand is whether to stick in an upcall. Perhaps methodFlags continues to have `is_end_training_trigger` but `Method` exports `has_upcall` which delegates (internally) to `is_end_training_trigger`.
>> 
>> When you get to the C++ code for the upcall, just proceed as you do already, with a comment that the upcall has been installed if and only if the method was marked for training end. Later on, that logic (again in C++) can be extended if we have other entry methods we wish to track. Or not; maybe we'll never have a second use for this mechanism. But even then, a more general looking name won't be harmful. And there is some value in keeping the CDS-specific logic more localized, in a handful of functions that handle "upcalls" whatever those might be.
>> 
>> There are other lightweight versions of upcalls floating around too, low-level method entry notifications. Some of them have been prototyped for Leyden (to trigger slow-burning recompilations from optimized code). Factoring this feature into an upcall pattern might make it easier to merge the various kinds of upcalls into a common facility, eventually.
>> 
>> The name `_end_training_predicate` promises more than it is; shouldn't it be `_end_training_count_limit`?
>> 
>> I suggest passing the `CompileCommandEnum` into `parse_method_pattern`. This will be a more user-facing instance of method patterns, so there will probably be special rules written for them. Probably more restrictive than the present rules for method patterns in other compile commands. For starters, we should choose between the dot-based and double-colon-based syntaxes. I can't predict which way this conversation will go, but it will happen and we will be glad to be able to contextualize the parse to this par...

@macarte The RuntimeUpcalls has support for method exit upcalls, but the changes in interpreter and compilers are missing. I guess this is intentional as there is no use-case for the method exit upcalls as of now.

-------------

PR Comment: https://git.openjdk.org/leyden/pull/21#issuecomment-2471589034

From macarte at openjdk.org  Thu Nov 14 20:01:38 2024
From: macarte at openjdk.org (Mat Carter)
Date: Thu, 14 Nov 2024 20:01:38 GMT
Subject: RFR: 8335358: [premain] Explore alternative ways to trigger the
 end of training run [v9]
In-Reply-To: <RZgymouejTif_mOAUQsvUGoRUPsk54W_xr89HYub1FY=.fdab65c5-d288-475d-9849-7b51ce51daed@github.com>
References: <RqVNoxdgqloUtbXxgiiqePA6XRrHkuolETkbzmzr7r4=.2b32af0e-69fa-48cf-89aa-322371ef9fb2@github.com>
 <u-mV8QEX6TlfZmz-t-yeFp7CJNjAQzyN9Qr_10NEu90=.778034a0-d6eb-44a8-8c8c-67a75cccff18@github.com>
 <d6t_EWfaQsUBe4qMFAQiKZmHZjtfhyTKH0LUCiVL1-Y=.f9a80e0f-8525-4c1c-84e8-bb80237fe07b@github.com>
 <RZgymouejTif_mOAUQsvUGoRUPsk54W_xr89HYub1FY=.fdab65c5-d288-475d-9849-7b51ce51daed@github.com>
Message-ID: <vBkF8hqoEQKm8Tw5bukT1RLIZMYBUp345krNz6U9kUM=.b934e0cc-8c0c-4ac9-95ea-a2224e9d3021@github.com>

On Tue, 5 Nov 2024 19:33:36 GMT, Mat Carter <macarte at openjdk.org> wrote:

>> Suggestion:  Rename end_training_check in the interpreter to method_entry_upcall.  That way, the very wide but shallow changes to the assembly code will support a variety of features, not just this particular one.
>> 
>> Probably the same thing would work for the JITs as well.  So when C1 says
>> 
>> 
>> CDSConfig::is_dumping_preimage_static_archive_with_triggers() && callee->is_end_training_trigger()
>> 
>> 
>> Basically, the assembler and interpreter and JITs should not be thinking about CDS, just this newfangled "upcall".
>> 
>> it might as well say `callee->has_entry_upcall()` and the CDS-specific logic can be moved nearer to the method classes.  The name `is_end_training_trigger` can be retired or used in fewer places, when the only question at hand is whether to stick in an upcall.  Perhaps methodFlags continues to have `is_end_training_trigger` but `Method` exports `has_upcall` which delegates (internally) to `is_end_training_trigger`.
>> 
>> When you get to the C++ code for the upcall, just proceed as you do already, with a comment that the upcall has been installed if and only if the method was marked for training end.  Later on, that logic (again in C++) can be extended if we have other entry methods we wish to track.  Or not; maybe we'll never have a second use for this mechanism.  But even then, a more general looking name won't be harmful.  And there is some value in keeping the CDS-specific logic more localized, in a handful of functions that handle "upcalls" whatever those might be.
>> 
>> There are other lightweight versions of upcalls floating around too, low-level method entry notifications.  Some of them have been prototyped for Leyden (to trigger slow-burning recompilations from optimized code).  Factoring this feature into an upcall pattern might make it easier to merge the various kinds of upcalls into a common facility, eventually.
>> 
>> The name `_end_training_predicate` promises more than it is; shouldn't it be `_end_training_count_limit`?
>> 
>> I suggest passing the `CompileCommandEnum` into `parse_method_pattern`.  This will be a more user-facing instance of method patterns, so there will probably be special rules written for them.  Probably more restrictive than the present rules for method patterns in other compile commands.  For starters, we should choose between the dot-based and double-colon-based syntaxes.  I can't predict which way this conversation will go, but it will happen and we will be glad to be able to contextualize the parse to...
>
>> Suggestion: Rename end_training_check in the interpreter to method_entry_upcall. That way, the very wide but shallow changes to the assembly code will support a variety of features, not just this particular one.
>> 
>> Probably the same thing would work for the JITs as well. So when C1 says
>> 
>> ```
>> CDSConfig::is_dumping_preimage_static_archive_with_triggers() && callee->is_end_training_trigger()
>> ```
>> 
>> Basically, the assembler and interpreter and JITs should not be thinking about CDS, just this newfangled "upcall".
>> 
>> it might as well say `callee->has_entry_upcall()` and the CDS-specific logic can be moved nearer to the method classes. The name `is_end_training_trigger` can be retired or used in fewer places, when the only question at hand is whether to stick in an upcall. Perhaps methodFlags continues to have `is_end_training_trigger` but `Method` exports `has_upcall` which delegates (internally) to `is_end_training_trigger`.
>> 
>> When you get to the C++ code for the upcall, just proceed as you do already, with a comment that the upcall has been installed if and only if the method was marked for training end. Later on, that logic (again in C++) can be extended if we have other entry methods we wish to track. Or not; maybe we'll never have a second use for this mechanism. But even then, a more general looking name won't be harmful. And there is some value in keeping the CDS-specific logic more localized, in a handful of functions that handle "upcalls" whatever those might be.
>> 
>> There are other lightweight versions of upcalls floating around too, low-level method entry notifications. Some of them have been prototyped for Leyden (to trigger slow-burning recompilations from optimized code). Factoring this feature into an upcall pattern might make it easier to merge the various kinds of upcalls into a common facility, eventually.
>> 
>> The name `_end_training_predicate` promises more than it is; shouldn't it be `_end_training_count_limit`?
>> 
>> I suggest passing the `CompileCommandEnum` into `parse_method_pattern`. This will be a more user-facing instance of method patterns, so there will probably be special rules written for them. Probably more restrictive than the present rules for method patterns in other compile commands. For starters, we should choose between the dot-based and double-colon-based syntaxes. I can't predict which way this conversation will go, but it will happen and we will be glad to be able to contextualize the parse to this par...

> @macarte The RuntimeUpcalls has support for method exit upcalls, but the changes in interpreter and compilers are missing. I guess this is intentional as there is no use-case for the method exit upcalls as of now.

That's correct, there's no use-case but I added the option so that a future implementation doesn't require a change to all existing upcalls.  I also added an assert that will trigger if an attempt to register an onMethodExit upcall is made.

-------------

PR Comment: https://git.openjdk.org/leyden/pull/21#issuecomment-2474929911

From macarte at openjdk.org  Thu Nov 14 20:01:39 2024
From: macarte at openjdk.org (Mat Carter)
Date: Thu, 14 Nov 2024 20:01:39 GMT
Subject: RFR: 8335358: [premain] Explore alternative ways to trigger the
 end of training run [v9]
In-Reply-To: <V9Z5Z73OzA5Drcng7Bwwz06vVZXlAiv8W8D-GjDzsQg=.baef1a5f-1d9d-4860-bbb3-cc27d4554a5a@github.com>
References: <RqVNoxdgqloUtbXxgiiqePA6XRrHkuolETkbzmzr7r4=.2b32af0e-69fa-48cf-89aa-322371ef9fb2@github.com>
 <u-mV8QEX6TlfZmz-t-yeFp7CJNjAQzyN9Qr_10NEu90=.778034a0-d6eb-44a8-8c8c-67a75cccff18@github.com>
 <d6t_EWfaQsUBe4qMFAQiKZmHZjtfhyTKH0LUCiVL1-Y=.f9a80e0f-8525-4c1c-84e8-bb80237fe07b@github.com>
 <RZgymouejTif_mOAUQsvUGoRUPsk54W_xr89HYub1FY=.fdab65c5-d288-475d-9849-7b51ce51daed@github.com>
 <V9Z5Z73OzA5Drcng7Bwwz06vVZXlAiv8W8D-GjDzsQg=.baef1a5f-1d9d-4860-bbb3-cc27d4554a5a@github.com>
Message-ID: <BIv5nrV85VdfwSWzXeS3iEs1dh9ka1sSFQQEJOFJ7To=.bcf48443-1b68-424b-a2cd-67d83f8dc533@github.com>

On Tue, 12 Nov 2024 21:10:18 GMT, Ashutosh Mehra <asmehra at openjdk.org> wrote:

>>> Suggestion: Rename end_training_check in the interpreter to method_entry_upcall. That way, the very wide but shallow changes to the assembly code will support a variety of features, not just this particular one.
>>> 
>>> Probably the same thing would work for the JITs as well. So when C1 says
>>> 
>>> ```
>>> CDSConfig::is_dumping_preimage_static_archive_with_triggers() && callee->is_end_training_trigger()
>>> ```
>>> 
>>> Basically, the assembler and interpreter and JITs should not be thinking about CDS, just this newfangled "upcall".
>>> 
>>> it might as well say `callee->has_entry_upcall()` and the CDS-specific logic can be moved nearer to the method classes. The name `is_end_training_trigger` can be retired or used in fewer places, when the only question at hand is whether to stick in an upcall. Perhaps methodFlags continues to have `is_end_training_trigger` but `Method` exports `has_upcall` which delegates (internally) to `is_end_training_trigger`.
>>> 
>>> When you get to the C++ code for the upcall, just proceed as you do already, with a comment that the upcall has been installed if and only if the method was marked for training end. Later on, that logic (again in C++) can be extended if we have other entry methods we wish to track. Or not; maybe we'll never have a second use for this mechanism. But even then, a more general looking name won't be harmful. And there is some value in keeping the CDS-specific logic more localized, in a handful of functions that handle "upcalls" whatever those might be.
>>> 
>>> There are other lightweight versions of upcalls floating around too, low-level method entry notifications. Some of them have been prototyped for Leyden (to trigger slow-burning recompilations from optimized code). Factoring this feature into an upcall pattern might make it easier to merge the various kinds of upcalls into a common facility, eventually.
>>> 
>>> The name `_end_training_predicate` promises more than it is; shouldn't it be `_end_training_count_limit`?
>>> 
>>> I suggest passing the `CompileCommandEnum` into `parse_method_pattern`. This will be a more user-facing instance of method patterns, so there will probably be special rules written for them. Probably more restrictive than the present rules for method patterns in other compile commands. For starters, we should choose between the dot-based and double-colon-based syntaxes. I can't predict which way this conversation will go, but it will happen and we will be glad to be able to contextualize th...
>
> @macarte The RuntimeUpcalls has support for method exit upcalls, but the changes in interpreter and compilers are missing. I guess this is intentional as there is no use-case for the method exit upcalls as of now.

Thanks @ashu-mehra for the review, I've addressed each of your comments (some with a code change)

-------------

PR Comment: https://git.openjdk.org/leyden/pull/21#issuecomment-2475060854

From iklam at openjdk.org  Thu Nov 14 20:01:43 2024
From: iklam at openjdk.org (Ioi Lam)
Date: Thu, 14 Nov 2024 20:01:43 GMT
Subject: RFR: 8335358: [premain] Explore alternative ways to trigger the
 end of training run [v11]
In-Reply-To: <eFkKB_LXKrDNgSj-EV7xXDo7JqNdhNNM93EVvI5VaT0=.4c7cd29e-f4e3-4430-9fda-f9fa784093e8@github.com>
References: <RqVNoxdgqloUtbXxgiiqePA6XRrHkuolETkbzmzr7r4=.2b32af0e-69fa-48cf-89aa-322371ef9fb2@github.com>
 <eFkKB_LXKrDNgSj-EV7xXDo7JqNdhNNM93EVvI5VaT0=.4c7cd29e-f4e3-4430-9fda-f9fa784093e8@github.com>
Message-ID: <SSpVXWGg43GdCL6ZfTKy0pX3JYW-zPjmKrtMH0Yt4ZY=.4b71ff55-a4d2-465c-ae24-a90c8b7b7e6c@github.com>

On Thu, 14 Nov 2024 19:57:14 GMT, Mat Carter <macarte at openjdk.org> wrote:

>> AOT training can be ended using either
>> 
>> - -XX: AOTEndTrainingOnMethodEntry =Hello.someMethod [same syntax as CompileOnly]
>> - -XX: AOTEndTrainingOnMethodEntry =Hello.someMethod,Hello.someOtherMethod,count=42
>> - jcmd <pid> AOT.end_training
>> 
>> supports arm64 and x64
>> 
>> note: the AOTEndTrainingOnMethodEntry is ignored when not recording
>> 
>> JBS Issue: https://bugs.openjdk.org/browse/JDK-8335358
>
> Mat Carter has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 43 additional commits since the last revision:
> 
>  - Enhanced upcall system testing options (based on PR feedback)
>  - Removing unused includes
>  - Remove cds include, no longer required after refactor
>  - merge issue
>  - fix merge issues
>  - Update other platforms
>  - Merge branch 'premain' into macarte-endtraining
>  - Merge pull request #2 from macarte/macarte-endtraining-refactor
>    
>    Macarte endtraining refactor
>  - minor edits
>  - small changes based on PR review
>  - ... and 33 more: https://git.openjdk.org/leyden/compare/4af49dbe...9c3cdf78

src/hotspot/share/cds/cdsEndTrainingUpcall.cpp line 86:

> 84: 
> 85: bool CDSEndTrainingUpcall::filter_method_callback(MethodDetails& methodDetails)
> 86: {

The HotSpot convention would be to use `method_details` instead of `methodDetails` for the names of fields, parameters and local variables.

See https://github.com/openjdk/jdk/blob/master/doc/hotspot-style.md#naming

-------------

PR Review Comment: https://git.openjdk.org/leyden/pull/21#discussion_r1841536917

From asmehra at openjdk.org  Thu Nov 14 20:01:42 2024
From: asmehra at openjdk.org (Ashutosh Mehra)
Date: Thu, 14 Nov 2024 20:01:42 GMT
Subject: RFR: 8335358: [premain] Explore alternative ways to trigger the
 end of training run [v11]
In-Reply-To: <eFkKB_LXKrDNgSj-EV7xXDo7JqNdhNNM93EVvI5VaT0=.4c7cd29e-f4e3-4430-9fda-f9fa784093e8@github.com>
References: <RqVNoxdgqloUtbXxgiiqePA6XRrHkuolETkbzmzr7r4=.2b32af0e-69fa-48cf-89aa-322371ef9fb2@github.com>
 <eFkKB_LXKrDNgSj-EV7xXDo7JqNdhNNM93EVvI5VaT0=.4c7cd29e-f4e3-4430-9fda-f9fa784093e8@github.com>
Message-ID: <653jthHOP-qMZ7iYAt4Wk_bb7njKohiPHJLLrqZ-THo=.f046aec0-ecd6-48c9-96ba-acf2dc67c3f0@github.com>

On Thu, 14 Nov 2024 19:57:14 GMT, Mat Carter <macarte at openjdk.org> wrote:

>> AOT training can be ended using either
>> 
>> - -XX: AOTEndTrainingOnMethodEntry =Hello.someMethod [same syntax as CompileOnly]
>> - -XX: AOTEndTrainingOnMethodEntry =Hello.someMethod,Hello.someOtherMethod,count=42
>> - jcmd <pid> AOT.end_training
>> 
>> supports arm64 and x64
>> 
>> note: the AOTEndTrainingOnMethodEntry is ignored when not recording
>> 
>> JBS Issue: https://bugs.openjdk.org/browse/JDK-8335358
>
> Mat Carter has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 43 additional commits since the last revision:
> 
>  - Enhanced upcall system testing options (based on PR feedback)
>  - Removing unused includes
>  - Remove cds include, no longer required after refactor
>  - merge issue
>  - fix merge issues
>  - Update other platforms
>  - Merge branch 'premain' into macarte-endtraining
>  - Merge pull request #2 from macarte/macarte-endtraining-refactor
>    
>    Macarte endtraining refactor
>  - minor edits
>  - small changes based on PR review
>  - ... and 33 more: https://git.openjdk.org/leyden/compare/4af49dbe...9c3cdf78

src/hotspot/share/cds/cdsConfig.cpp line 675:

> 673: 
> 674: bool CDSConfig::is_dumping_preimage_static_archive_with_triggers() {
> 675:   return _is_dumping_static_archive && CacheDataStore != nullptr && CDSPreimage == nullptr && !FLAG_IS_DEFAULT(AOTEndTrainingOnMethodEntry);

This can be replaced with `is_dumping_preimage_static_archive() && !FLAG_IS_DEFAULT(AOTEndTrainingOnMethodEntry)`

src/hotspot/share/cds/cdsEndTrainingUpcall.hpp line 33:

> 31: 
> 32: #include "code/codeBlob.hpp"
> 33: #include "code/vmreg.hpp"

umm...do we need all these includes here? I think some may be redundant.
Same applies to runtimeUpcallNop.hpp.

src/hotspot/share/runtime/runtimeUpcallNop.cpp line 45:

> 43: bool RuntimeUpcallNop::filter_method_callback(MethodDetails& methodDetails)
> 44: {
> 45:   return false;

Shouldn't this be returning true to enable it for all methods as per the documentation for AddRuntimeUpcallsNOP in globals.hpp?

src/hotspot/share/runtime/runtimeUpcalls.cpp line 96:

> 94:   // which upcalls to call, but it would be more efficient than the current implementation as we'd avoid the
> 95:   // method matching and simply map bits to indexes.
> 96: 

Another way to speed it up is to store a hashset of Method pointers in each RuntimeUpcallInfo when `RuntimeUpcalls::mark_for_upcalls` is called for the method. Then the method matching can be replaced with a  lookup in the hashset.

src/hotspot/share/runtime/runtimeUpcalls.hpp line 39:

> 37: };
> 38: 
> 39: typedef void (*RuntimeUpcall)(JavaThread* current);

Not needed for the AOTEndTrainingOnMethodEntry option, but I think passing the method as the argument for which upcall is being done could be beneficial.

src/hotspot/share/runtime/runtimeUpcalls.hpp line 65:

> 63: 
> 64: public:
> 65:   static RuntimeUpcallInfo* Create(const char* upcallName, const RuntimeUpcall upcall, const RuntimeUpcallMethodFilterCallback methodFilter) {

I think the convention is to always use lower case for methods, so probably s/Create/create

-------------

PR Review Comment: https://git.openjdk.org/leyden/pull/21#discussion_r1838771681
PR Review Comment: https://git.openjdk.org/leyden/pull/21#discussion_r1838771772
PR Review Comment: https://git.openjdk.org/leyden/pull/21#discussion_r1838777872
PR Review Comment: https://git.openjdk.org/leyden/pull/21#discussion_r1838772148
PR Review Comment: https://git.openjdk.org/leyden/pull/21#discussion_r1838784474
PR Review Comment: https://git.openjdk.org/leyden/pull/21#discussion_r1838772055

From macarte at openjdk.org  Thu Nov 14 20:01:44 2024
From: macarte at openjdk.org (Mat Carter)
Date: Thu, 14 Nov 2024 20:01:44 GMT
Subject: RFR: 8335358: [premain] Explore alternative ways to trigger the
 end of training run [v11]
In-Reply-To: <653jthHOP-qMZ7iYAt4Wk_bb7njKohiPHJLLrqZ-THo=.f046aec0-ecd6-48c9-96ba-acf2dc67c3f0@github.com>
References: <RqVNoxdgqloUtbXxgiiqePA6XRrHkuolETkbzmzr7r4=.2b32af0e-69fa-48cf-89aa-322371ef9fb2@github.com>
 <eFkKB_LXKrDNgSj-EV7xXDo7JqNdhNNM93EVvI5VaT0=.4c7cd29e-f4e3-4430-9fda-f9fa784093e8@github.com>
 <653jthHOP-qMZ7iYAt4Wk_bb7njKohiPHJLLrqZ-THo=.f046aec0-ecd6-48c9-96ba-acf2dc67c3f0@github.com>
Message-ID: <e6lScO9YXyQ_TaF7MjWX0b59_70KZv2jlcBdNzWnN_w=.bf8ee2fc-afc9-401f-b3d0-e7d9e973dfae@github.com>

On Tue, 12 Nov 2024 21:12:21 GMT, Ashutosh Mehra <asmehra at openjdk.org> wrote:

>> Mat Carter has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 43 additional commits since the last revision:
>> 
>>  - Enhanced upcall system testing options (based on PR feedback)
>>  - Removing unused includes
>>  - Remove cds include, no longer required after refactor
>>  - merge issue
>>  - fix merge issues
>>  - Update other platforms
>>  - Merge branch 'premain' into macarte-endtraining
>>  - Merge pull request #2 from macarte/macarte-endtraining-refactor
>>    
>>    Macarte endtraining refactor
>>  - minor edits
>>  - small changes based on PR review
>>  - ... and 33 more: https://git.openjdk.org/leyden/compare/4af49dbe...9c3cdf78
>
> src/hotspot/share/runtime/runtimeUpcallNop.cpp line 45:
> 
>> 43: bool RuntimeUpcallNop::filter_method_callback(MethodDetails& methodDetails)
>> 44: {
>> 45:   return false;
> 
> Shouldn't this be returning true to enable it for all methods as per the documentation for AddRuntimeUpcallsNOP in globals.hpp?

Thanks for pointing that out; I've updated the AddRuntimeUpcallsNOP to a string that is of the format "=[onMethodEntry|onMethodExit]:[all|none]" allowing us to test (a) mulitple upcalls ,(b) test onMethodExit (that it currently asserts), (c) have the NOP upcall do nothing, but either install an upcall on no methods or all methods

> src/hotspot/share/runtime/runtimeUpcalls.hpp line 39:
> 
>> 37: };
>> 38: 
>> 39: typedef void (*RuntimeUpcall)(JavaThread* current);
> 
> Not needed for the AOTEndTrainingOnMethodEntry option, but I think passing the method as the argument for which upcall is being done could be beneficial.

I can see where that might be useful, but it's complicated as we have a single call back but the 'method' comes back from the interpreter as a ciMethod instance and from c1/c2 as a Method instance, so we'd need to either have a level of indirection so we could wrap the method (like I did with MethodDetails) or have multiple callbacks

-------------

PR Review Comment: https://git.openjdk.org/leyden/pull/21#discussion_r1841260379
PR Review Comment: https://git.openjdk.org/leyden/pull/21#discussion_r1841263686

From macarte at openjdk.org  Thu Nov 14 20:01:44 2024
From: macarte at openjdk.org (Mat Carter)
Date: Thu, 14 Nov 2024 20:01:44 GMT
Subject: RFR: 8335358: [premain] Explore alternative ways to trigger the
 end of training run [v11]
In-Reply-To: <e6lScO9YXyQ_TaF7MjWX0b59_70KZv2jlcBdNzWnN_w=.bf8ee2fc-afc9-401f-b3d0-e7d9e973dfae@github.com>
References: <RqVNoxdgqloUtbXxgiiqePA6XRrHkuolETkbzmzr7r4=.2b32af0e-69fa-48cf-89aa-322371ef9fb2@github.com>
 <eFkKB_LXKrDNgSj-EV7xXDo7JqNdhNNM93EVvI5VaT0=.4c7cd29e-f4e3-4430-9fda-f9fa784093e8@github.com>
 <653jthHOP-qMZ7iYAt4Wk_bb7njKohiPHJLLrqZ-THo=.f046aec0-ecd6-48c9-96ba-acf2dc67c3f0@github.com>
 <e6lScO9YXyQ_TaF7MjWX0b59_70KZv2jlcBdNzWnN_w=.bf8ee2fc-afc9-401f-b3d0-e7d9e973dfae@github.com>
Message-ID: <maQMc2DuD7YOqdqiD8y1Lo7wTblTSclHAxABKFPPHSk=.499671a8-c634-4399-a6d0-be89ff237447@github.com>

On Wed, 13 Nov 2024 22:18:31 GMT, Mat Carter <macarte at openjdk.org> wrote:

>> src/hotspot/share/runtime/runtimeUpcallNop.cpp line 45:
>> 
>>> 43: bool RuntimeUpcallNop::filter_method_callback(MethodDetails& methodDetails)
>>> 44: {
>>> 45:   return false;
>> 
>> Shouldn't this be returning true to enable it for all methods as per the documentation for AddRuntimeUpcallsNOP in globals.hpp?
>
> Thanks for pointing that out; I've updated the AddRuntimeUpcallsNOP to a string that is of the format "=[onMethodEntry|onMethodExit]:[all|none]" allowing us to test (a) mulitple upcalls ,(b) test onMethodExit (that it currently asserts), (c) have the NOP upcall do nothing, but either install an upcall on no methods or all methods

So now the filter will use the value from the string 'none' (false) or 'all' (true) for the filter result

-------------

PR Review Comment: https://git.openjdk.org/leyden/pull/21#discussion_r1841261213

From macarte at openjdk.org  Thu Nov 14 21:10:57 2024
From: macarte at openjdk.org (Mat Carter)
Date: Thu, 14 Nov 2024 21:10:57 GMT
Subject: RFR: 8335358: [premain] Explore alternative ways to trigger the
 end of training run [v12]
In-Reply-To: <RqVNoxdgqloUtbXxgiiqePA6XRrHkuolETkbzmzr7r4=.2b32af0e-69fa-48cf-89aa-322371ef9fb2@github.com>
References: <RqVNoxdgqloUtbXxgiiqePA6XRrHkuolETkbzmzr7r4=.2b32af0e-69fa-48cf-89aa-322371ef9fb2@github.com>
Message-ID: <DlgmxPHHWxj4awdv2Vhnee-aAxChx5vtC9v5BGxr1Rs=.5501e9b4-1d5c-438c-91fc-ea8cc6715621@github.com>

> AOT training can be ended using either
> 
> - -XX: AOTEndTrainingOnMethodEntry =Hello.someMethod [same syntax as CompileOnly]
> - -XX: AOTEndTrainingOnMethodEntry =Hello.someMethod,Hello.someOtherMethod,count=42
> - jcmd <pid> AOT.end_training
> 
> supports arm64 and x64
> 
> note: the AOTEndTrainingOnMethodEntry is ignored when not recording
> 
> JBS Issue: https://bugs.openjdk.org/browse/JDK-8335358

Mat Carter has updated the pull request incrementally with one additional commit since the last revision:

  Fix platform dependent compiler error

-------------

Changes:
  - all: https://git.openjdk.org/leyden/pull/21/files
  - new: https://git.openjdk.org/leyden/pull/21/files/9c3cdf78..129eb018

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=leyden&pr=21&range=11
 - incr: https://webrevs.openjdk.org/?repo=leyden&pr=21&range=10-11

  Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/leyden/pull/21.diff
  Fetch: git fetch https://git.openjdk.org/leyden.git pull/21/head:pull/21

PR: https://git.openjdk.org/leyden/pull/21

From ioi.lam at oracle.com  Thu Nov 14 21:48:14 2024
From: ioi.lam at oracle.com (ioi.lam at oracle.com)
Date: Thu, 14 Nov 2024 13:48:14 -0800
Subject: AOT caching of WeakReference
Message-ID: <03b06d00-76d9-4c7a-8b94-a9231c19bcac@oracle.com>

Here's my? experiment with storing weak references in the AOT cache

https://github.com/iklam/jdk/blob/8341587-example-of-weakref-problem-in-aot-cache/AOT-Cache-And-WeakRef.md

As discussed in the Leyden meeting today, there seem to be at least two 
issues

- In premain, CDS keeps every archived heap object alive (look for 
UsePermanentHeapObjects in the code). I plan to get rid of this shortly. 
UsePermanentHeapObjects is intended for the AOT compiler to access heap 
objects, but it should be implemented differently (only remember objects 
that are required by AOT code, which should include only Strings and 
class mirrors).

- WeakReference.queue is not pointing to the production-time value of 
ReferenceQueue.NULL


Thanks

- Ioi


From john.r.rose at oracle.com  Thu Nov 14 22:15:22 2024
From: john.r.rose at oracle.com (John Rose)
Date: Thu, 14 Nov 2024 14:15:22 -0800
Subject: JEP 483 targeted to upcoming JDK 24 release
Message-ID: <2B254364-F54A-4063-9D6E-BF593A2EBC05@oracle.com>

Our first ?premain? feature, affectionally called ?JEP One?,
is now targeted[1].

[1] https://mail.openjdk.org/pipermail/jdk-dev/2024-November/009645.html

Congratulations to everyone who helped reach this important milestone!

JEP One will provide a firm foundation for a number of interesting
future JEPs, as seen in our EA.  I think Ioi will be preparing a
final version of the PR, which we will re-review and re-test quickly,
and then push during the next day or two.  The conflict level is
small (just one conflict with Lilliput) so we are hopeful the rebase
will work smoothly.

There is some technical debt here to follow up on.  We will be
fixing a small handful of simple JCK failures which we know about,
and any other bugs that crop up at at level P3 or higher.

As a bigger step, we will be simplifying some parts of the
changes for JEP One.  Many simplifications will probably land
in JDK 25.  For example, workarounds avoiding SoftReference
usage will be removed after we get the kinks out of using
SoftReference in the assembly phase (AP).  And the AP heap
walking logic can be made simpler and more robust.

There is a long list of possible cleanups in JDK-8343023[2].

[2] https://bugs.openjdk.org/browse/JDK-8343023

Our big user visible items, probably follow-up JEPs,
will include a simplified workflow (make the AOT config
generation transparent, optionally), persisting of
method profiles from the training run (TR), and persisting
of compiled method code (the ?JIT cache?).  All of these
are visible in the Early Access builds, and were demonstrated
last year at JVMLS 2023.

As an independent workflow RFE, I want to get Mat Carter?s
end-of-training hook pushed whenever we think it is ready.
That is https://bugs.openjdk.org/browse/JDK-8335358.  It
is useful whether the workflow as a whole is improved or
kept as it is today.

I am also hoping to clean up and rationalize the boot
sequence of the JVM, the initialization of the First Hundred
classes.  (They are about a hundred, not exactly 100.)  This
is likely to involve more explicit declarations of ordering
conditions inside the JDK source code itself.  We also want
to split one class C into two CA+CR so that C can be
initialized at assembly time and CR can be initialized
at production runtime.  Instead of CR, we might also
double down on runtime-setup methods in CA, but that
precludes the use of final fields in CA, since finals
in CA would be initialized by the assembly phase, and
not revisited at startup.  We are avoiding magic that
makes final variables change, so if a variable needs
to change, it cannot be a final field.  It can be
stable, though; we expect to amend the contract of
stable variables to allow changes during the assembly
phase.

Thanks,

? John

From erik.osterlund at oracle.com  Thu Nov 14 22:51:46 2024
From: erik.osterlund at oracle.com (Erik Osterlund)
Date: Thu, 14 Nov 2024 22:51:46 +0000
Subject: AOT caching of WeakReference
In-Reply-To: <03b06d00-76d9-4c7a-8b94-a9231c19bcac@oracle.com>
References: <03b06d00-76d9-4c7a-8b94-a9231c19bcac@oracle.com>
Message-ID: <0AB70696-6324-4D16-8E37-E1AC50C88334@oracle.com>

Hi Ioi,

You can also get caught with a WR with discovered fields and next fields populated. They are mutated by the GC. They can drag in random unrelated stuff into your object graph that you might not want to stuff into the archive.

/Erik

> On 14 Nov 2024, at 22:48, ioi.lam at oracle.com wrote:
> 
> ?Here's my  experiment with storing weak references in the AOT cache
> 
> https://github.com/iklam/jdk/blob/8341587-example-of-weakref-problem-in-aot-cache/AOT-Cache-And-WeakRef.md
> 
> As discussed in the Leyden meeting today, there seem to be at least two issues
> 
> - In premain, CDS keeps every archived heap object alive (look for UsePermanentHeapObjects in the code). I plan to get rid of this shortly. UsePermanentHeapObjects is intended for the AOT compiler to access heap objects, but it should be implemented differently (only remember objects that are required by AOT code, which should include only Strings and class mirrors).
> 
> - WeakReference.queue is not pointing to the production-time value of ReferenceQueue.NULL
> 
> 
> Thanks
> 
> - Ioi
> 

From john.r.rose at oracle.com  Fri Nov 15 04:12:19 2024
From: john.r.rose at oracle.com (John Rose)
Date: Fri, 15 Nov 2024 04:12:19 +0000
Subject: AOT caching of WeakReference
In-Reply-To: <0AB70696-6324-4D16-8E37-E1AC50C88334@oracle.com>
References: <03b06d00-76d9-4c7a-8b94-a9231c19bcac@oracle.com>
 <0AB70696-6324-4D16-8E37-E1AC50C88334@oracle.com>
Message-ID: <0B0A601D-260D-4547-AA0F-93D1C51DDC45@oracle.com>

Right. Is there any way to deterministically normalize or clear those links? Maybe we could unlink them at dump and relink at startup. 

> On Nov 14, 2024, at 2:52?PM, Erik Osterlund <erik.osterlund at oracle.com> wrote:
> 
> ?Hi Ioi,
> 
> You can also get caught with a WR with discovered fields and next fields populated. They are mutated by the GC. They can drag in random unrelated stuff into your object graph that you might not want to stuff into the archive.
> 
> /Erik
> 
>> On 14 Nov 2024, at 22:48, ioi.lam at oracle.com wrote:
>> 
>> ?Here's my  experiment with storing weak references in the AOT cache
>> 
>> https://github.com/iklam/jdk/blob/8341587-example-of-weakref-problem-in-aot-cache/AOT-Cache-And-WeakRef.md
>> 
>> As discussed in the Leyden meeting today, there seem to be at least two issues
>> 
>> - In premain, CDS keeps every archived heap object alive (look for UsePermanentHeapObjects in the code). I plan to get rid of this shortly. UsePermanentHeapObjects is intended for the AOT compiler to access heap objects, but it should be implemented differently (only remember objects that are required by AOT code, which should include only Strings and class mirrors).
>> 
>> - WeakReference.queue is not pointing to the production-time value of ReferenceQueue.NULL
>> 
>> 
>> Thanks
>> 
>> - Ioi
>> 

From erik.osterlund at oracle.com  Fri Nov 15 10:14:03 2024
From: erik.osterlund at oracle.com (Erik Osterlund)
Date: Fri, 15 Nov 2024 10:14:03 +0000
Subject: AOT caching of WeakReference
In-Reply-To: <0B0A601D-260D-4547-AA0F-93D1C51DDC45@oracle.com>
References: <03b06d00-76d9-4c7a-8b94-a9231c19bcac@oracle.com>
 <0AB70696-6324-4D16-8E37-E1AC50C88334@oracle.com>
 <0B0A601D-260D-4547-AA0F-93D1C51DDC45@oracle.com>
Message-ID: <F75EDF3A-743D-40C4-9AFC-9DC021F4A1EC@oracle.com>

I would probably rather ensure that the CDS code that copies object payload into the archived heap, does not copy discovered fields, hence leaving them blank (null). As for next fields, finding non-null values would typically imply that the referent has just been cleared by the GC, and the reference just hasn?t been processed yet by its registered queue, to perform some cleanup. This could mean we are about to snapshot state that hasn?t been cleaned up yet and hence contains garbage state you probably don?t want to linger around to the next run. Perhaps it would be better to only allow snapshotted WRs to be registered with the NULL queue, and treat the next fields the same as discovered fields: don?t copy them to the archived heap.

/Erik

> On 15 Nov 2024, at 05:12, John Rose <john.r.rose at oracle.com> wrote:
> 
> Right. Is there any way to deterministically normalize or clear those links? Maybe we could unlink them at dump and relink at startup. 
> 
>> On Nov 14, 2024, at 2:52?PM, Erik Osterlund <erik.osterlund at oracle.com> wrote:
>> 
>> ?Hi Ioi,
>> 
>> You can also get caught with a WR with discovered fields and next fields populated. They are mutated by the GC. They can drag in random unrelated stuff into your object graph that you might not want to stuff into the archive.
>> 
>> /Erik
>> 
>>> On 14 Nov 2024, at 22:48, ioi.lam at oracle.com wrote:
>>> 
>>> ?Here's my  experiment with storing weak references in the AOT cache
>>> 
>>> https://github.com/iklam/jdk/blob/8341587-example-of-weakref-problem-in-aot-cache/AOT-Cache-And-WeakRef.md
>>> 
>>> As discussed in the Leyden meeting today, there seem to be at least two issues
>>> 
>>> - In premain, CDS keeps every archived heap object alive (look for UsePermanentHeapObjects in the code). I plan to get rid of this shortly. UsePermanentHeapObjects is intended for the AOT compiler to access heap objects, but it should be implemented differently (only remember objects that are required by AOT code, which should include only Strings and class mirrors).
>>> 
>>> - WeakReference.queue is not pointing to the production-time value of ReferenceQueue.NULL
>>> 
>>> 
>>> Thanks
>>> 
>>> - Ioi
>>> 


From macarte at openjdk.org  Fri Nov 15 22:16:28 2024
From: macarte at openjdk.org (Mat Carter)
Date: Fri, 15 Nov 2024 22:16:28 GMT
Subject: RFR: 8335358: [premain] Explore alternative ways to trigger the
 end of training run [v13]
In-Reply-To: <RqVNoxdgqloUtbXxgiiqePA6XRrHkuolETkbzmzr7r4=.2b32af0e-69fa-48cf-89aa-322371ef9fb2@github.com>
References: <RqVNoxdgqloUtbXxgiiqePA6XRrHkuolETkbzmzr7r4=.2b32af0e-69fa-48cf-89aa-322371ef9fb2@github.com>
Message-ID: <-ivsI4_VW6pE9BvjrNGCwpJ7mBwLU6ruriJDfeJ04o8=.b56144bc-7618-495e-8049-7403dfa8ad77@github.com>

> AOT training can be ended using either
> 
> - -XX: AOTEndTrainingOnMethodEntry =Hello.someMethod [same syntax as CompileOnly]
> - -XX: AOTEndTrainingOnMethodEntry =Hello.someMethod,Hello.someOtherMethod,count=42
> - jcmd <pid> AOT.end_training
> 
> supports arm64 and x64
> 
> note: the AOTEndTrainingOnMethodEntry is ignored when not recording
> 
> JBS Issue: https://bugs.openjdk.org/browse/JDK-8335358

Mat Carter has updated the pull request incrementally with one additional commit since the last revision:

  fixed whitespace and nonnull issue

-------------

Changes:
  - all: https://git.openjdk.org/leyden/pull/21/files
  - new: https://git.openjdk.org/leyden/pull/21/files/129eb018..2e44c252

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=leyden&pr=21&range=12
 - incr: https://webrevs.openjdk.org/?repo=leyden&pr=21&range=11-12

  Stats: 6 lines in 2 files changed: 1 ins; 0 del; 5 mod
  Patch: https://git.openjdk.org/leyden/pull/21.diff
  Fetch: git fetch https://git.openjdk.org/leyden.git pull/21/head:pull/21

PR: https://git.openjdk.org/leyden/pull/21

From duke at openjdk.org  Tue Nov 19 03:35:36 2024
From: duke at openjdk.org (duke)
Date: Tue, 19 Nov 2024 03:35:36 GMT
Subject: git: openjdk/leyden: premain: Fixed
 CDSAccess::get_archived_object_permanent_index() to only keep alive archived
 oops that have a permanent index (previously every archived oop was kept
 alive)
Message-ID: <4087dd30-5101-4d27-8a57-470df5529bee@openjdk.org>

Changeset: ac533a7e
Branch: premain
Author:    iklam <ilam3388 at gmail.com>
Date:      2024-11-18 17:20:45 +0000
URL:       https://git.openjdk.org/leyden/commit/ac533a7e29810400fd6ccbbeab01e478dfd2f9ac

Fixed CDSAccess::get_archived_object_permanent_index() to only keep alive archived oops that have a permanent index (previously every archived oop was kept alive)

! src/hotspot/share/cds/archiveBuilder.cpp
! src/hotspot/share/cds/archiveHeapLoader.cpp
! src/hotspot/share/cds/archiveHeapLoader.hpp
! src/hotspot/share/cds/archiveHeapWriter.cpp
! src/hotspot/share/cds/filemap.cpp
! src/hotspot/share/cds/heapShared.cpp
! src/hotspot/share/cds/heapShared.hpp
! src/hotspot/share/cds/metaspaceShared.cpp
! src/hotspot/share/code/SCCache.cpp


From ioi.lam at oracle.com  Tue Nov 19 06:51:05 2024
From: ioi.lam at oracle.com (ioi.lam at oracle.com)
Date: Mon, 18 Nov 2024 22:51:05 -0800
Subject: AOT caching of WeakReference
In-Reply-To: <03b06d00-76d9-4c7a-8b94-a9231c19bcac@oracle.com>
References: <03b06d00-76d9-4c7a-8b94-a9231c19bcac@oracle.com>
Message-ID: <ad41bda6-cef2-4848-877c-068e6fc845e6@oracle.com>


On 11/14/24 1:48 PM, ioi.lam at oracle.com wrote:
> Here's my? experiment with storing weak references in the AOT cache
>
> https://github.com/iklam/jdk/blob/8341587-example-of-weakref-problem-in-aot-cache/AOT-Cache-And-WeakRef.md 
>
>
> As discussed in the Leyden meeting today, there seem to be at least 
> two issues
>
> - In premain, CDS keeps every archived heap object alive (look for 
> UsePermanentHeapObjects in the code). I plan to get rid of this 
> shortly. UsePermanentHeapObjects is intended for the AOT compiler to 
> access heap objects, but it should be implemented differently (only 
> remember objects that are required by AOT code, which should include 
> only Strings and class mirrors).
>

I have fixed UsePermanentHeapObjects so that only objects that are 
referenced by AOT-compiled methods are kept alive.

See 
https://git.openjdk.org/leyden/commit/ac533a7e29810400fd6ccbbeab01e478dfd2f9ac



From macarte at openjdk.org  Tue Nov 19 21:33:34 2024
From: macarte at openjdk.org (Mat Carter)
Date: Tue, 19 Nov 2024 21:33:34 GMT
Subject: RFR: 8335358: [premain] Explore alternative ways to trigger the
 end of training run [v14]
In-Reply-To: <RqVNoxdgqloUtbXxgiiqePA6XRrHkuolETkbzmzr7r4=.2b32af0e-69fa-48cf-89aa-322371ef9fb2@github.com>
References: <RqVNoxdgqloUtbXxgiiqePA6XRrHkuolETkbzmzr7r4=.2b32af0e-69fa-48cf-89aa-322371ef9fb2@github.com>
Message-ID: <pMVWHJsKD2GybBbPrJ9K5oZgiSMhH1rMu8MqSSrM0wY=.2f6cc1ac-274b-40d3-af67-518adf78f8a8@github.com>

> AOT training can be ended using either
> 
> - -XX: AOTEndTrainingOnMethodEntry =Hello.someMethod [same syntax as CompileOnly]
> - -XX: AOTEndTrainingOnMethodEntry =Hello.someMethod,Hello.someOtherMethod,count=42
> - jcmd <pid> AOT.end_training
> 
> supports arm64 and x64
> 
> note: the AOTEndTrainingOnMethodEntry is ignored when not recording
> 
> JBS Issue: https://bugs.openjdk.org/browse/JDK-8335358

Mat Carter has updated the pull request incrementally with one additional commit since the last revision:

  Support for builds without CDS

-------------

Changes:
  - all: https://git.openjdk.org/leyden/pull/21/files
  - new: https://git.openjdk.org/leyden/pull/21/files/2e44c252..b47655e4

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=leyden&pr=21&range=13
 - incr: https://webrevs.openjdk.org/?repo=leyden&pr=21&range=12-13

  Stats: 14 lines in 4 files changed: 10 ins; 4 del; 0 mod
  Patch: https://git.openjdk.org/leyden/pull/21.diff
  Fetch: git fetch https://git.openjdk.org/leyden.git pull/21/head:pull/21

PR: https://git.openjdk.org/leyden/pull/21

From macarte at openjdk.org  Wed Nov 20 19:00:23 2024
From: macarte at openjdk.org (Mat Carter)
Date: Wed, 20 Nov 2024 19:00:23 GMT
Subject: RFR: 8335358: [premain] Explore alternative ways to trigger the
 end of training run [v15]
In-Reply-To: <RqVNoxdgqloUtbXxgiiqePA6XRrHkuolETkbzmzr7r4=.2b32af0e-69fa-48cf-89aa-322371ef9fb2@github.com>
References: <RqVNoxdgqloUtbXxgiiqePA6XRrHkuolETkbzmzr7r4=.2b32af0e-69fa-48cf-89aa-322371ef9fb2@github.com>
Message-ID: <buyr9_DE8yvXU_J70o-qkGFdx-wVRKelPZ_O1zEwZPM=.ff1f3fcb-d15c-4c88-8402-2a5f5bb7fe56@github.com>

> AOT training can be ended using either
> 
> - -XX: AOTEndTrainingOnMethodEntry =Hello.someMethod [same syntax as CompileOnly]
> - -XX: AOTEndTrainingOnMethodEntry =Hello.someMethod,Hello.someOtherMethod,count=42
> - jcmd <pid> AOT.end_training
> 
> supports arm64 and x64
> 
> note: the AOTEndTrainingOnMethodEntry is ignored when not recording
> 
> JBS Issue: https://bugs.openjdk.org/browse/JDK-8335358

Mat Carter has updated the pull request incrementally with one additional commit since the last revision:

  replace camelCase with lowercase_lowercase

-------------

Changes:
  - all: https://git.openjdk.org/leyden/pull/21/files
  - new: https://git.openjdk.org/leyden/pull/21/files/b47655e4..c3e9d4e4

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=leyden&pr=21&range=14
 - incr: https://webrevs.openjdk.org/?repo=leyden&pr=21&range=13-14

  Stats: 158 lines in 14 files changed: 21 ins; 12 del; 125 mod
  Patch: https://git.openjdk.org/leyden/pull/21.diff
  Fetch: git fetch https://git.openjdk.org/leyden.git pull/21/head:pull/21

PR: https://git.openjdk.org/leyden/pull/21

From macarte at openjdk.org  Wed Nov 20 19:00:29 2024
From: macarte at openjdk.org (Mat Carter)
Date: Wed, 20 Nov 2024 19:00:29 GMT
Subject: RFR: 8335358: [premain] Explore alternative ways to trigger the
 end of training run [v11]
In-Reply-To: <SSpVXWGg43GdCL6ZfTKy0pX3JYW-zPjmKrtMH0Yt4ZY=.4b71ff55-a4d2-465c-ae24-a90c8b7b7e6c@github.com>
References: <RqVNoxdgqloUtbXxgiiqePA6XRrHkuolETkbzmzr7r4=.2b32af0e-69fa-48cf-89aa-322371ef9fb2@github.com>
 <eFkKB_LXKrDNgSj-EV7xXDo7JqNdhNNM93EVvI5VaT0=.4c7cd29e-f4e3-4430-9fda-f9fa784093e8@github.com>
 <SSpVXWGg43GdCL6ZfTKy0pX3JYW-zPjmKrtMH0Yt4ZY=.4b71ff55-a4d2-465c-ae24-a90c8b7b7e6c@github.com>
Message-ID: <mYAtgjm2YmRB3DX9Sry0RhTjkhTFn5fSCwfN4x8zeDE=.fa67e7ee-0df9-410d-9eac-0bb772be1189@github.com>

On Thu, 14 Nov 2024 04:16:10 GMT, Ioi Lam <iklam at openjdk.org> wrote:

>> Mat Carter has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 43 additional commits since the last revision:
>> 
>>  - Enhanced upcall system testing options (based on PR feedback)
>>  - Removing unused includes
>>  - Remove cds include, no longer required after refactor
>>  - merge issue
>>  - fix merge issues
>>  - Update other platforms
>>  - Merge branch 'premain' into macarte-endtraining
>>  - Merge pull request #2 from macarte/macarte-endtraining-refactor
>>    
>>    Macarte endtraining refactor
>>  - minor edits
>>  - small changes based on PR review
>>  - ... and 33 more: https://git.openjdk.org/leyden/compare/afefb1ec...9c3cdf78
>
> src/hotspot/share/cds/cdsEndTrainingUpcall.cpp line 86:
> 
>> 84: 
>> 85: bool CDSEndTrainingUpcall::filter_method_callback(MethodDetails& methodDetails)
>> 86: {
> 
> The HotSpot convention would be to use `method_details` instead of `methodDetails` for the names of fields, parameters and local variables.
> 
> See https://github.com/openjdk/jdk/blob/master/doc/hotspot-style.md#naming

Thanks, I've corrected all camelCase to lowercase_lowercase format

-------------

PR Review Comment: https://git.openjdk.org/leyden/pull/21#discussion_r1850832635

From vlivanov at openjdk.org  Wed Nov 20 19:13:49 2024
From: vlivanov at openjdk.org (Vladimir Ivanov)
Date: Wed, 20 Nov 2024 19:13:49 GMT
Subject: RFR: 8335358: [premain] Explore alternative ways to trigger the
 end of training run [v15]
In-Reply-To: <buyr9_DE8yvXU_J70o-qkGFdx-wVRKelPZ_O1zEwZPM=.ff1f3fcb-d15c-4c88-8402-2a5f5bb7fe56@github.com>
References: <RqVNoxdgqloUtbXxgiiqePA6XRrHkuolETkbzmzr7r4=.2b32af0e-69fa-48cf-89aa-322371ef9fb2@github.com>
 <buyr9_DE8yvXU_J70o-qkGFdx-wVRKelPZ_O1zEwZPM=.ff1f3fcb-d15c-4c88-8402-2a5f5bb7fe56@github.com>
Message-ID: <ZWA7GABBtU6KOk5Qv_87wtH_V6dua9FnRvjY4M8SJr4=.a8f9a304-7e98-486e-bf50-9e8fcaf5414d@github.com>

On Wed, 20 Nov 2024 19:00:23 GMT, Mat Carter <macarte at openjdk.org> wrote:

>> AOT training can be ended using either
>> 
>> - -XX: AOTEndTrainingOnMethodEntry =Hello.someMethod [same syntax as CompileOnly]
>> - -XX: AOTEndTrainingOnMethodEntry =Hello.someMethod,Hello.someOtherMethod,count=42
>> - jcmd <pid> AOT.end_training
>> 
>> supports arm64 and x64
>> 
>> note: the AOTEndTrainingOnMethodEntry is ignored when not recording
>> 
>> JBS Issue: https://bugs.openjdk.org/browse/JDK-8335358
>
> Mat Carter has updated the pull request incrementally with one additional commit since the last revision:
> 
>   replace camelCase with lowercase_lowercase

Nice work, Mat! 

Speaking of proposed API, there's a lot of complexity around `-XX:AOTEndTrainingOnMethodEntry=`. What's the primary use case for it? Does it add much for uncooperative case (no application changes) compared to `jcmd`?

For cooperative case, a dedicated method in JDK an application can call seems more appropriate.

-------------

PR Comment: https://git.openjdk.org/leyden/pull/21#issuecomment-2489353011

From iklam at openjdk.org  Thu Nov 21 17:38:48 2024
From: iklam at openjdk.org (Ioi Lam)
Date: Thu, 21 Nov 2024 17:38:48 GMT
Subject: RFR: 8335358: [premain] Explore alternative ways to trigger the
 end of training run [v15]
In-Reply-To: <ZWA7GABBtU6KOk5Qv_87wtH_V6dua9FnRvjY4M8SJr4=.a8f9a304-7e98-486e-bf50-9e8fcaf5414d@github.com>
References: <RqVNoxdgqloUtbXxgiiqePA6XRrHkuolETkbzmzr7r4=.2b32af0e-69fa-48cf-89aa-322371ef9fb2@github.com>
 <buyr9_DE8yvXU_J70o-qkGFdx-wVRKelPZ_O1zEwZPM=.ff1f3fcb-d15c-4c88-8402-2a5f5bb7fe56@github.com>
 <ZWA7GABBtU6KOk5Qv_87wtH_V6dua9FnRvjY4M8SJr4=.a8f9a304-7e98-486e-bf50-9e8fcaf5414d@github.com>
Message-ID: <kioi4yYILjx34ujFCGPxNw2VUe0FPXCHvCnMySbHT5k=.41d73d75-9ecb-4286-8476-f55b84fd8573@github.com>

On Wed, 20 Nov 2024 19:10:56 GMT, Vladimir Ivanov <vlivanov at openjdk.org> wrote:

> Nice work, Mat! 
> 
> 
> 
> Speaking of proposed API, there's a lot of complexity around `-XX:AOTEndTrainingOnMethodEntry=`. What's the primary use case for it? Does it add much for uncooperative case (no application changes) compared to `jcmd`?
> 
> 
> 
> For cooperative case, a dedicated method in JDK an application can call seems more appropriate.

I think this PR has its benefits:

- JCMD triggers are imprecise. If you using jcmd to trigger the end of training based on time, or based on parsing the app's log, it may capture too much or too little profile.

- An API would require modifying the app.

So this PR allows precise timing  (for apps that have a well-known Java method that's called during important state transition) without modifying the app itself.

-------------

PR Comment: https://git.openjdk.org/leyden/pull/21#issuecomment-2491867215

From macarte at openjdk.org  Fri Nov 22 00:20:01 2024
From: macarte at openjdk.org (Mat Carter)
Date: Fri, 22 Nov 2024 00:20:01 GMT
Subject: RFR: 8335358: [premain] Explore alternative ways to trigger the
 end of training run [v16]
In-Reply-To: <RqVNoxdgqloUtbXxgiiqePA6XRrHkuolETkbzmzr7r4=.2b32af0e-69fa-48cf-89aa-322371ef9fb2@github.com>
References: <RqVNoxdgqloUtbXxgiiqePA6XRrHkuolETkbzmzr7r4=.2b32af0e-69fa-48cf-89aa-322371ef9fb2@github.com>
Message-ID: <855JTsU8to4JZ0deTZO7sYrF3s9jVDms5L6Smswe06c=.e6b1168b-a5de-404c-98d8-6df5d9b90b75@github.com>

> AOT training can be ended using either
> 
> - -XX: AOTEndTrainingOnMethodEntry =Hello.someMethod [same syntax as CompileOnly]
> - -XX: AOTEndTrainingOnMethodEntry =Hello.someMethod,Hello.someOtherMethod,count=42
> - jcmd <pid> AOT.end_training
> 
> supports arm64 and x64
> 
> note: the AOTEndTrainingOnMethodEntry is ignored when not recording
> 
> JBS Issue: https://bugs.openjdk.org/browse/JDK-8335358

Mat Carter has updated the pull request incrementally with one additional commit since the last revision:

  fixing include issue

-------------

Changes:
  - all: https://git.openjdk.org/leyden/pull/21/files
  - new: https://git.openjdk.org/leyden/pull/21/files/c3e9d4e4..88200565

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=leyden&pr=21&range=15
 - incr: https://webrevs.openjdk.org/?repo=leyden&pr=21&range=14-15

  Stats: 9 lines in 2 files changed: 4 ins; 4 del; 1 mod
  Patch: https://git.openjdk.org/leyden/pull/21.diff
  Fetch: git fetch https://git.openjdk.org/leyden.git pull/21/head:pull/21

PR: https://git.openjdk.org/leyden/pull/21

From duke at openjdk.org  Fri Nov 22 05:17:21 2024
From: duke at openjdk.org (duke)
Date: Fri, 22 Nov 2024 05:17:21 GMT
Subject: git: openjdk/leyden: created branch premain-temp-merge-after-jep-483
 based on the branch master containing 495 unique commits
Message-ID: <57cc32bc-076d-4186-a521-a6a04e0ae636@openjdk.org>

The following commits are unique to the premain-temp-merge-after-jep-483 branch:
========================================================
626b62aa: Initial commit
f5321dd8: Merge branch 'master' of https://github.com/openjdk/leyden into premain
d47df36e: More relocInfo fixes for aarch64
49d035dc: Additional card table fixes
3daab89b: 8314078: HotSpotConstantPool.lookupField() asserts due to field changes in ConstantPool.cpp
6825ecde: another relocInfo patching fix
e6b1a385: Disabled some CDS test code affected by the earleir work-around of JDK-8307468 in leyden-premain
e0562723: Added information in the CDS map file for the permobj segments used by HeapShared::get_archived_object_permanent_index()
678d7f91: Added test/hotspot/jtreg/premain/javac_helloworld start-up benchmark
ff3c770b: Temp work-around for some test cases that see two classes named "jdk.proxy2.$Proxy8"
78194355: Skip SC generation for custom class loaders
f3073178: Merge branch 'master' of https://github.com/openjdk/leyden into premain
cc41a022: 8314846: Do not store Klass::_secondary_super_cache in CDS archive
1d1854ed: fixed missing JvmtiExport::_should_notify_object_alloc address when AOT compiling javac with -XX:+ArchiveInvokeDynamic
631e2f0f: improve logging of (cds+resolve)
980b5d92: Updated field CP archiving to match JDK-8301996
4ef082a9: 8314247: JVMCI: expected int64_t but JavaThread::_held_monitor_count is of type intx (this is temp workaround from the withdrawn https://github.com/openjdk/jdk/pull/15295)
e1a591e7: Check recompilation schedule existence to avoid potential crashes if the
3ed4a9e3: ConstantPool::remove_unshareable_info() clean up
3910f790: Fixed premain docs
38387c5b: Fix MethodTrainingData::make
9b92dfc2: removed unused SystemDictionaryShared::init_dumptime_info(Method* m)
d1b61107: 8315431: ArchiveHeapWriter::get_filler_size_at() cannot handle buffer expansion
fa813084: Renames in cached code.
fd82682f: Rename class SCArchive to SCCache
d960fb15: Work around for JDK-8315719: [premain] -XX:+ArchiveInvokeDynamic is incompatible with old classes
265c226c: Fixed CDS jtreg tests to use new command line options like -XX:+StoreCachedCode
7b4a856f: cds: archive unique exception instances
c95d6058: New "one step training run" workflow with -XX:CacheDataStore
d1048945: Leyden-premain + spring-petclinic demo
c5b1038e: fixed typos in tests
a8280594: Fix processing cache file path
f4a8e829: Merge branch 'master' of https://github.com/openjdk/leyden into premain
0628708c: Fixed CDS dump-time crash when running spring-petclinic
cb990ebb: Merge branch 'premain' of https://github.com/openjdk/leyden into premain
b3d60142: Enable StoreCachedCode/LoadCachedCode by default for new "1-step" workflow. Only String.charAt() is compiled.
878f9211: Replace UseNewCode with Xlog
a5a03fcf: Reduce upcalls into Java code when preloading classes
2b430286: Fixed bug where nested init of hidden classes may cause the JIT see unloaded classes
d9dfd990: Slightly reduce L2 code load failures
e54e70eb: Fix aarch64 ic_stuc_code_size when SCC is used
052f03ab: Initialize classes with no <clinit> methods at class preloading time
3fb808ee: Cleaned up -Xlog:scc to print to the specified LogStream instead of tty
6457b720: disabled preresolution of invokehandle due to crashes in petclinit
cddf20a5: Record and verify VM configuration
9193e092: Work-around JDK-8316802: unresolved classes may be countered by compiler
006a379f: Force some java.lang.invoke classes to be pre-initialized at dump time
2cd2c1d6: Skip not entrant preload code
397ffe78: 8316994: Avoid modifying ClassLoader and Module objects during -Xshare:dump
805509e4: Avoid archiving lambda proxies of interfaces that have clinit, as that can change program execution order.
c8f40024: 8316802: unresolved classes may be encountered by compiler
751985bb: added ClassPrelinker::replay_training_at_init_for_javabase_preloaded_classes()
5fdb6150: Concurrect training replay
ba3525df: Merge branch 'master' of https://github.com/openjdk/leyden into premain
7beb8afb: added SA support for TrainingReplayThread
66472a00: Add more initiated classes to preloaded dictionaries
d7aaf280: 8317269: Preload all verification dependencies for archived classes
fc7a5714: Fix windows build (part1)
0ddd91d7: Fix some compile errors
b88c089e: CDSPreimage: avoid buffer overflow when composing command line
4c252ff1: Fixed petclinit crash with -XX:CacheDataStore
f2dcff59: fixed -Xlog:cds+resolve=debug crash when dumping dynamic archive
329f7083: Added perf counter for ClassPrelinker
9fbe454b: Partial fix for minimal build failure (minimal build is still not working)
824e20a9: Temp work around for JDK-8317841: do not archive spring's dynamically generated classes
d8f230d6: Improve -XX:+TraceBytecodes output
e6ffcc14: ClassPrelinker: Handle LambdaMetafactory::altMetafactory BSM
d5d5675a: ciInstanceKlass::print_impl() should not print to tty
73757d06: Print CodeCache contents
77d438dd: Fixed crash when perf counters were used without UsePerfData
ebd5cd99: Fixed crash with spring-clinic caused by the JDK-8317841 work-around
36ec4386: Merge remote-tracking branch 'origin/premain' into premain
fdbbfbe5: Trace class initialization
6be1da15: Improve logging
41c18f3f: Use :: in class::methods output
e604a480: Include recompilation_schedule in -XX:+PrintSharedArchiveAndExit output
2b0a2761: Improve TrainingData validation
9d46f861: Fixed "guarantee(tag_at(which).is_klass()) failed: Corrupted constant pool" when running SpringRest.java
a7b1d41d: Proper support for -XX:+RecordTraining for new workflow
5d601b75: Merge remote-tracking branch 'origin/premain' into premain
7a1ac31b: Archive all MethodCounters and MDOs w/ -XX:+ForceProfiling
3367220a: C2: full clinit barrier support
df4778d6: SCC: Update stub lists
a270cf73: Enable compiler directives for *::print (workaround)
5dfa2e3c: Profile VM locks usages on main application thread
9d14caf5: StressClassInitBarriers
6a7ed51c: Enable ProfileVMLocks with -Xlog:init
8f6dc80e: added jdk/test/lib/LeydenTester class for writing jtreg tests for old and new workflow
55aeef9d: fixed no-pch build
499e5ee5: added problem list for JDK-8318142
8ca4331e: too much -Xlog:scc*=trace caused IndyTestBase.java to crash; change to -Xlog:scc
d40ed355: Fixed tier 1: serviceability/jvmti/RedefineClasses/RedefineDoubleDelete.java
aa64bdbc: Fixed some JVMCI test failures
85a1cb67: load archived training data from preimage, so AOT code can be generated during final dump
49be0722: Populate shared code cache during terminal stage
31b2e9c8: ProfileVMCalls, ProfileRuntimeCalls, and misc minor logging enhancements
a72c1abf: Support CDSAccess::get_archived_object_permanent_index() in new workflow
f285fa40: Temp workaround for JDK-8318463 so we see the full error log when a build fails
0995df58: new workflow: transfer the training data from training run to final CDS image
7fe44552: fixed macos build; added more cases in appcds/leyden/ExcludedClasses.java
48369197: reformatted cdsConfig.hpp, to be upstreamed to mainline
82f357fd: Merge branch 'master' of https://github.com/openjdk/leyden into premain
8ddc8999: New workflow - partial fix for cleaning up excluded classes in CDS preimage
ded30038: clean up CDS code
09c307a9: improved LeydenTester and move the old-workflow test cases to XxxOldWF.java
a20e93ae: Added spring-petclinic as jtreg test
daf1c352: Fixed handlig of unregistered classes in new workflow -- now they are passed from preimage to final image
2ac33fb0: Fixed crash in new workflow during AOT compilation in ciEnv::compute_init_state_for_precompiled()
0ea8bcc7: Memoize requests to ciMethod::inline_instructions_size()
04b7722e: 8317841: [premain] do not archive spring's dynamically generated classes
940808ea: fixed gcc build
d124175e: Merge remote-tracking branch 'origin/premain' into premain
3d685701: cds: Archive Class::reflectionData
4868039e: ArchiveReflectionData: fix misc bugs
5f6cf611: ProfileVMLocks et al: minor fixes
c3b8a2d6: Introduce per-queue locks for compiler queues
ad04f75f: x86: build fixes
8d8e7ffb: UseNewCode -> PrintTrainingInfo
a2fe5863: refactor CDS static dumping to prepare for writing cached code into archive
2223ded5: prototype for AOT support API in CDS
24b1f71b: Merge branch 'master' of https://github.com/openjdk/leyden into premain
cf888902: Avoid calling maybe_compile_early() in CompilationPolicy::compile_if_required()
941b7395: Merge branch 'master' of https://github.com/openjdk/leyden into premain
355721f4: Fix a comment
1b6bbf7c: Merge branch 'premain' of https://github.com/openjdk/leyden into premain
cae8bbcd: Added CDSConfig::preserve_all_dumptime_verification_states() to control whether classes verified by the old verifier can be archived
6956205d: Enabled -XX:+ArchiveReflectionData testing for old workflow
5b425c99: Merge branch 'master' of https://github.com/openjdk/leyden into premain
53f58b4a: Fix out of order locking assert
03e75986: profiling: allocate counters only when required
4f132c3f: perf: reserve enough space for perf counters
dae99ee7: interpreter: fast class initialization checks
c1b25c18: Make PrintBytecodeHistogram a diagnostic flag
2cfa3f9b: Conditionally disable cached code (per tier/level)
fa117d9a: Partially lock-free compile queues
9fafb52e: misc logging cleanups
de8c5f7f: Improve ProfileRuntimeCalls
28202ede: ProfileVMCalls: MethodHandleNative support
6fc2b196: ProfileRuntimeCalls: per-bytecode type stats
f5c9ac2f: fixed comments
d5f46a2a: Delay profiling (level 3) compilation of cold methods
a323049d: Added HeapShared::initialize_default_subgraph_classes() to fix crashes related to ArchiveReflectionData
b6da82f7: Need to link all classes first in HeapShared::initialize_default_subgraph_classes()
1df6ffc7: Improve asserts: Dump diagnostic info for unloaded classes
b260a58d: Compiler queue statistics
e6e3d998: Fix a race on Method::_code
603b991f: minor test updates
01fdf6cb: Merge remote-tracking branch 'refs/remotes/origin/premain' into premain
e6e31107: Archive static field CP entries
01728040: Archive static method CP entries
5419a467: cleaned up usage of CDSConfig for new workflow
c2b8b34a: Merge branch 'master' of https://github.com/openjdk/leyden into premain
6ac83f8b: Merge branch 'premain' of https://github.com/openjdk/leyden into premain
a083bba5: Merge remote-tracking branch 'refs/remotes/origin/premain' into premain
0a6d86bc: Disable can_call_java restriction assert during prelinkage
c874a89b: Fast clinit barrier support check for getstatic/putstatic
aa4b88e7: Include CP entry tag name in diagnostic messages
b1d06f4d: Archive virtual method CP entries
fb3c95db: Archive method CP entries for invokespecial
b0580852:  Archive interface method CP entries
5fc06cbb: Archive invokehandle CP entries
2f4883a0: Improve nmethod dependency validation logging
5a7347ef: Bytecode resolution tracing
6ae7b1bc: support ArchiveReflectionData in new workflow
f6fb9874: fixed comment in test/hotspot/jtreg/premain/javac_helloworld/run.sh
6a1dae6b: Delay SCCache::intialize until CDS archives are loaded
317ce587: moved the dumping of CDS archive in new workflow to after all VM initialization has finished
01070b76: Fixed bug where full module graph is not archived for new workflow
32ff6f10: Implemented -XX:+ArchiveDynamicProxies
fd6e803a: new brnchmark framework for running very short running programs like javac
f609b8cc: Merge branch 'master' of https://github.com/openjdk/leyden into premain
ffb0c323: Validate preload code before publishing
448178f4: preimage: Don't tweak execution mode
3f419ca9: Remove CompileTrainingData::_top_method
a1f8fada: Don't tweak execution mode when dumping final static archive
d3cf79d2: preload: Link holder klass before preloading code
851add9e: Improve precompilation in one-step training mode
ec8dd9af: Use normal thresholds for methods without TD
6586489a: Upgraded test/hotspot/jtreg/premain/spring-petclinic to 3.2.0
7faed7fc: Fix input checks in Vector API intrinsics
838db4ae: Fix C1/C2-only execution modes
614cc684: Don't archive resolved continuation intrinsic methods.
c21af5eb: print timing stats in test/hotspot/jtreg/runtime/cds/appcds/applications/javac/JavacBenchApp.java
96997843: updated test/hotspot/jtreg/runtime/cds/appcds/leyden/SpringPetClinic.java to use version 3.2.0
02bc149e: updated SpringPetClinic.java jtreg test to add -Dspring.aot.enabled=true
59f97f7a: Refactored Leyden test supporting code to jdk.test.lib.cds.CDSAppTester
e73de845: Factor in should_delay()
98e5def2: Merge remote-tracking branch 'origin/premain' into premain
8b33d432: Replace SCExclude with IgnorePrecompiled/DontPreload compiler directives
171b36be: 8324975: Add a timer to track thread cpu time
0f133e24: Cleanup stage 1
132e17bf: Cleanup stage 2
53ae54b2: Add utility classes to measure thread cpu time when profiling runtime
22e0cdad: Fix style
2a6bb953: Style
bf07660e: Merge remote-tracking branch 'origin/master' into premain-leyden
a09b607d: Fix merge
912673ad: Bump PerfDataMemorySize to 64K
8e14c78d: Improve deoptimization statistics
26287c89: -Xlog:init: Print CodeCache and SCCache statistics
752d8ff7: Print statistics on nmethod usage
db5a3a48: Profile VM operations on main thread
bad62d8b: CountBytecodesPerThread
74dc8e81: Improve MutexLocker profiling
6411b580: Disable thread info dumps with -Xlog:init
f1b73aa8: Not-yet-loaded methods encountered in MDOs
66c2dd52: Archive class name w/ -XX:+ArchiveReflectionData
03bb7f8a: Fix build after merge
30167b6a: Cleanup MethodHandle perf counters
0adf2bfb: Fix build on linux
20c228d3: Warn about disarmed nmethods
9ffe8dbf: Guard counter usages with UsePerfData
02e1d310: Properly handle deopts on entry when gathering trap statistics
69923449: Upgrade bytecode counter to 64-bit
b58e0e37: Add negative cache for built-in loaders
393f79de: Persist loader negative cache in CDS archive
361fc8b0: Positive lookup cache for built-in loaders and some cleanup
94d92c34: Improve compiler thread wakeup
d5338d4e: Fix formatting
30014130: Merge remote-tracking branch 'upstream/master' into premain-upstream
24f8953d: Merge fixes
23420d8c: Prelink shared classes
073d304f: Merge branch 'premain' into premain-upstream
45dd3f92: Merge remote-tracking branch 'upstream/master' into premain-upstream
62a344e2: Fixed #include problems for PerfTickCounters::elapsed_counter_value_ms(), etc
33a9ad6e: Comment out the assert for safepoint check when iterating classes in DumpTimeSharedClassTable
2fc639a0: Merge branch 'premain' into premain-upstream
a672b4c6: Fix x86 build
68a9c37e: Create a new DumpTimeSharedClassTable iterator for the classes loaded by builtin loaders
13ba0651: Fix a bug in writing contents of loader's negative lookup cache
f1d6e862: Automate warmup testing using springboot-petclinic
23baf7f1: Update comments in WarmupMakefile
b99380d4: Patch file for springboot petclinic application used by WarmupMakefile
5ce5a63d: Refactored code for deciding what classes can be pre-inited
3cdb6324: Fixed test failure in ArchiveHeapTestClass.java
d67819c6: 8318064: [premain] CDS regenerated LambdaForm holder classes miss some methods
240a6782: jmeter test plan used by WarmupMakefile for springboot-petclinic
50ea3bb5: Minor improvements to WarmupMakefile
e22e1cb3: Trace method invocation from interpreter
c3984a7c: relocInfo et al tracing improvements
e2d3cd2b: fix Windows build failure
1303cece: Merge remote-tracking branch 'origin/premain' into premain
69027b63: Fix allocation types
5012dd1b: Remove init tracking
4e9483db: Improve CompileTrainingData::_init_deps_left handling
0a5d7374: Remove TD factories for symbols
8461779d: Remove forgotten trap
29f450bf: Clear the pending_queue_processed flag on Method when it is removed from
18b0133d: Remove hard-coded path to JMeter binary
13505f2c: Remove unnecessary CTD::make()
9e34721c: Control tracing of thread time in Perf counters using a command line
8045ce6e: Fix a bug to set initial value of PerfCounter to 0
bcb9a07d: Regenerate lambda form invoker classes at the end of preimage run. This reduces the number of LambdaForm classes in final image.
ade0acb9: -Xlog:scc+codecache: Print archive code info on exit
89e5c3b0: Detailed statistics on deoptimization events
42e27ba6: Rework compiler init dependency tracking
30374ac3: Improve TrainingData verification
c3436de3: Improve constant pool preresolution
1ec8eb15: Archive array classes
93f1bb69: SSCache cleanups
3e7e1ecb: Code precompilation support
2cc64803: Avoid code archiving failures due to code buffer resizing on AArch64
7cbabcf3: precompilation: Improve class init state computation in CI
a4d97992: Disable PreloadSharedClasses by default (same as JEP)
7afec36f: Merge branch 'master' of https://github.com/openjdk/leyden into premain
11038ecc: Reverted unnecessary -XX:-PreloadSharedClasses changes to CDS test cases
de041670: fixed some minor jtreg test failures
bc34228f: UsePermanentHeapObjects should be enabled for static dump only if PreloadSharedClasses==true.
a2b89415: remove TestResolvedJavaMethod.java test from ProblemList-premain.txt
c1344d31: Use non-throwing metaspace allocation for training data
439fe8f1: Merge branch 'premain' of github.com:openjdk/leyden into premain
4ffe4f8a: split classPreloader.cpp out of classPrelinker.cpp
83a96356: split classPreloader.cpp out of classPrelinker.cpp (step2)
b25ef0a5: Added "make mainline_vs_premain" to get simple comparisons of pet clinic
9ace2f21: specify all CDS optimizations explicitly in pet clinic makefile
de0cf7b9: Added "make compare_premain_builds" and "make diff_training_logs" for pet-clinic
d9366c89: Fix handling of array class (ObjArrayKlass) of TypeArrayKlass
e17d2f5a: When -XX:+PreloadSharedClasses is given, enable all CDS optimizations that depends on it
bd80f5b8: Fixed "Booted and returned" timing in spring-petclinic to also include time spent in the JVM before main() method is executed
b091912a: 8328563: Store java.lang.ClassLoader::{packages, package2certs} into CDS archive
76adcbf1: Merge branch 'master' of https://github.com/openjdk/leyden into premain
b31eddf5: Save and relocate stop msg strings in cached code on aarch64
85c0eea0: Remove InstanceKlass::_training_data.
c93abe3f: Fixed macosx/aarch64 build
dab044f9: Do not write @cp for classes that are regenerated during the JDK build
2003a807: Simplified archiving of Universe:*_exception_instance() objects
e2902a0e: Quick preloading of classes for the boot loader without calling SystemDictionary::load_instance_class()
6c0c6ff3: Cleaned up ClassLinker APIs for pre-resolution of ConstantPool entries
b298683e: Merge branch 'master' of https://github.com/openjdk/leyden into premain
a81c1cc9: Fixed crashing with https://github.com/sdeleuze/petclinic-efficient-container
5c1ad75f: Merge branch 'master' into premain
b3933d8b: more clean up of pre-resolution of CP entries
b2bc2555: 8329721: Store java.lang.Boolean.{TRUE,FALSE} in CDS archive
0217a396: 8329728: Read arbitrarily long lines in ClassListParser
aa6d5444: Minor fixes to WarmupMakefile
7ecd661f: Cleanup remove_unshareable_info() and restore_unshareable_info()
908503f1: (1) enable ArchiveLoaderLookupCache, by default when PreloadSharedClasses is enabled; (2) do not disable Inline for new workflow
057360f9: Add targets to run warmup test using new single-step workflow
338719d4: Refactoring: moved class init code into cds/classPreinitializer.cpp
f546e292: More refactoring: move code related to pre-initialization to classPreinitializer.cpp
fdcc1180: Refactor: move is_vm_class() into ClassPreloader; fixed inconsistent use of xxx_klass vs _xxx_class in ClassPreloader/ClassPrelinker
e211d5d7: Moved the management of the "receipes" to the FinalImageRecipes class
194ae8ad: Added build/test scripts for new demo: https://helidon.io/docs/v4/se/guides/quickstart
a98bb060: Add missing reference traversal
4b1f9091: Remove chaining of CTDs in a list. Capture only the ones we need.
5f276f58: Remove unused allocate() functions
14902cb2: Added build/test scripts for new demo: https://quarkus.io/guides/getting-started
c9feeb6f: Merge branch 'master' of https://github.com/openjdk/leyden into premain
2e3f59b9: Allocate all TD in the null class loader data
48d45275: Fix warning
811bad88: 8321944: fix PcDesc cache initialization
64987121: Original code unpacked from https://guides.micronaut.io/latest/creating-your-first-micronaut-app-maven-java.zip , downloaded on 2024/04/22
9fa97221: Added "Micronaut First App" as a test case
04a95521: Added "make bench" in all four framework demos; added relative performance numbers to premain/README.md
c40b2993: Fixed graphs in MD file
74c0bfad: minor benchmark tweaks
8571089b: 8331086: TrainingData allocation from Metaspace may fail
5a39b00d: Merge branch 'master' of https://github.com/openjdk/leyden into premain
192edefb: set CachedCodeMaxSize to 512M when StoreCachedCode is enabled
aa66815f: Improved remove_unshareable_info for TrainingData (to avoid race condition where compiler thread might be adding more TD when we are collecting them ourside of the CDS safepoint)
039c0219: Added instructions for running demos
12ab8469: Improved benchmarks docs; added "make compare_premain_builds"
ba07d82a: Merge branch 'premain' of https://github.com/openjdk/leyden into premain
674deded: Copy contents of ProblemList-premain.txt to ProblemList.txt, in preparation of regular tiers1-3 testing
9076b74d: Disable CDS if runtime VM options are incompatible with archives dumped with -XX:+PreloadSharedClasses
d44cdcbf: No need to call TrainingData::restore_all_unshareable_info
9e106c59: Added instructions for running Leyden-specific jtreg tests
eb25dca8: Avoid grabbing locks inside MethodData::extra_data_lock
aee254fa: Merge branch 'master' into premain
ba8bf2e7: Merge branch 'master' of https://github.com/openjdk/leyden into premain
49baa911: Fixed typo in test/hotspot/jtreg/premain/lib/build-for-jtreg.sh
84afb72e: Remove restore_unshareable_info()
86ec27a6: Fixed _compare_premain_builds in test/hotspot/jtreg/premain/lib/Bench.gmk to be compatible with premain/spring-petclinic/Makefile
f11f4604: support benchmarking for new workflow
9c7ef4ea: include missing TASKSET arg
b2e204aa: Refactored applications/JavacBench.java test; added sanity validation
3674fc32: Clean up ClassListParser
06232a3c: prepare for upstreaming JDK-8293980
cd261d16: Merge branch 'master' of https://github.com/openjdk/leyden into premain
d85a7e89: fixed merge
b8b686a2: make/data/hotspot-symbols/symbols-unix is no longer needed (removed in mainline)
15f51139: Add snapshotting mechanism for training data
8dee825a: Moved Leyden overview docs to $REPO/README.md
32811147: Fixed typo and added disclaimers
4faa7202: Added checks for G1GC during new workflow training run
f7d6f17a: Removed Leyden-specific workaround in test cases
7e9d14f0: 8332340: Add JavacBench as a test case for CDS
c67940c7: 8333222: Allow SerialGC and ParallelGC to be used in Leyden training run
9c138a8e: Convert not fatal log_warning(scc) to log_info(scc)
739a39bf: (a) InterpreterRuntime::cds_resolve_invoke does not need mh param; (b) merged some changes from https://github.com/openjdk/jdk/pull/19355
ada67450: (1) Added UseCompatibleCompressedOops flag to allow AOT code to work for all heap ranges; (2) updated README.md for "same collector" limitation
485dee94: Enable -Xshare:on by default for new workflow -- so people know immediately if the Leyden optimizations are disabled due to a misconfiguration
9834890b: Fixed failure in ExcludedClassesOldWF.java caused by the previous commit
5214ff1e: Fixed -XX:+UseCompatibleCompressedOops crash on macos
789f1452: Added notes about supported garbage collectors
74171dd1: Treat SCC warnings as error when -Xshare:on
16e4da7f: Disabled tests for differnt coop encodings
0518a5d2: Proper handling of pending stale tasks in CompileQueue
abdbf473: Editorial changes to README, plus better mermaid format from lujaniuk
d0981826: Merge branch 'master' of https://github.com/openjdk/leyden into premain
5d856130: small clean up in README.md
e4c038df: Clarify that this repo contains only code from the "premain" prototype
5d3441c4: Hand-merged some changes from JDK-8293980 (in preparation for mainline merge)
29503891: Merge commit 'eb2488fd1781af49d936348d5f75731de2006ce7' into premain
d9e797b3: reduce the use of is_on; add missing class linking statistics
8dc7f15d: Merge branch 'master' of https://github.com/openjdk/leyden into premain
8f2eb149: Factor out recompilation logic
e2b46a20: Merge branch 'premain' of github.com:openjdk/leyden into premain
ce790adc: remove leftover commented code during last merge
173181cc: Added test case for 8317269: [premain] Archive old classes in verified state when CDSLoadedClasses is enabled
ce3ce7aa: Removed incorrect message "Cannot dump shared archive while using shared archive"
d415c65b: Instead of using ArchiveInvokeDynamic directly, use CDSConfig::is_dumping_invokedynamic()
8716f47e: Merge branch 'premain' of github.com:openjdk/leyden into premain
2bb2fbc4: Always set TD::_holder.
5ebf8d6c: Stabilize find() behavior during snapshot
3726f1df: Cleanup
159c6693: Simplify KTD constructor
dc69d066: 8334421: assert(!oldbox->is_unbalanced()) failed: this should not be called for unbalanced region
5b27856a: 8335220: C2: Missing check for Opaque4 node in EscapeAnalysis
c27385e3: 8335221: Some C2 intrinsics incorrectly assume that type argument is compile-time constant
517e4b23: Merge branch 'premain-ea' of https://github.com/openjdk/leyden into premain
098c309d: 8335492: Some hidden classes are not archived for ArchiveInvokeDynamic
bd601c50: 8309634: Resolve CONSTANT_MethodRef at CDS dump time
d2446866: fixed windows build
40373c0a: Merge branch 'master' of https://github.com/openjdk/leyden into premain
43e77c8e: Added test case for 8328563: Store java.lang.ClassLoader::{packages, package2certs} into CDS archive
d9e3e5fe: 8335804: ArchiveInvokeDynamic causes BootstrapMethodError
77e9f24c: Print more info about command-line to help debugging the dumping process
c49bd56b: Guard JVMTI related code with INCLUDE_JVMTI
5fdfef1d: Added more test case for JDK-8317269: Archive old classes in verified state when CDSLoadedClasses is enabled
dbdccb37: 8335790: Store protection domains into CDS archive
f65cccf9: 8336033: [premain] assert(comp != nullptr) when using SCCache with -XX:TieredStopAtLevel=1
5545aa0c: Add option to omit archive heap verification
79329d86: Resolve merge conflicts
77eff765: Added Pet Clinic benchmark to track performance of JEP-8315737
ded7dae0: Work-around JDK-8336414 which causes failure in ResolvedConstants.java
f519a8a9: Removed incorrect message "Cannot dump shared archive while using shared archive"
c2a5c9ac: 8335804: ArchiveInvokeDynamic causes BootstrapMethodError
4c385b55: 8336033: [premain] assert(comp != nullptr) when using SCCache with -XX:TieredStopAtLevel=1
d47557ad: Cache few stubs on aarch64 similar to x64 to avoid empty cached code
1b665a99: 8336144: assert(CDSConfig::is_dumping_invokedynamic()) when ZGC is used during archive creation
7ae716cf: Merge branch 'premain-ea' of https://github.com/openjdk/leyden into premain
b949235f: Add missing far branch checks on aarch64 to generate trampolines
34484563: Fixed CDS failures in tier3 (-XX:+UseSerialGC, -XX:+UseZGC, etc)
f1c370f9: More fixes for CDS failures in tier3 (-XX:+UseZGC, etc)
baad503c: 8336562: Quit CacheDataStore creation when incompatible module options are specified
35a48278: Merge branch 'master' of https://github.com/openjdk/leyden into premain
974f53b6: SCC: Handle ConstantDynamic CP entries
ed32488c: C2: Fix upcall into compiler runtime w/ -XX:ClassInitBarrierMode=2
a03a63e3: Merge branch 'premain' of https://github.com/openjdk/leyden into premain
8423515c: Change TD has table to have keys with direct references
ec5eb996: Clean up cdsEnumKlass.cpp code for upstreaming to mainline
b24e6fa2: New "AOT" options (-XX:AOTCache=xxx, etc) implemented as aliases to existing CDS options
37b332cc: fixed typo
f972b8dc: Added -XX:AOTMode as specified in latest JEP draft. Removed RecordAOTConfiguration/CreateAOTCache
c0826c59: Fixed silly mistake in last commit
d7c97f22: Added spring-boot-getting-started demo
1e18a09d: Refactoring of ClassPreloader and ClassPrelinker. Now we have AOTClassLinker, AOTConstantPoolResolver and AOTLinkedClassBulkLoader
ee11cf1b: Exclude test cases that cannot be executed with -Dtest.dynamic.cds.archive=true
12225130: Added -XX:AOTMode=on/off/auto
dce3658c: Cleanup: added CDSConfig::is_using_aot_linked_classes()
9a4ea6ff: preload classes tests do not work with ZGC
d9811b4a: Do not use decomp count for one step workflow
c01a9bce: fixed windows build
8b148497: renamed -XX:+PreloadSharedClasses to -XX:+AOTClassLinking
0d054eb4: 8337965: Add null check to ClaimMetadataVisitingOopIterateClosure::do_cld()
0bcbbff9: more clean up on new AOTxxx command-line flags
82c7ad0d: 8338011: CDS archived heap object support for 64-bit Windows
9f24a918: Ignore decomp counter for one-step. Cleanup in prints.
65bd17a2: More AOTClassLinking clean up
5ca207db: Merge branch 'master' of https://github.com/openjdk/leyden into premain
b065ce74: Set _sc_count correctly when only one of c1 compiler or c2 compiler is enabled
e4059458: Enable finding scc entry when _sc_count is 0 (using -XX:-UseCodeLoadThread)
2158a6a5: jcheck configuration for Leyden
864228c7: Update JEP titles, and add a general warning that flags will be renamed
a5daf90b: Fixed runtime/cds/DeterministicDump.java failure on Windows
1ee287c0: (1) Fixed Windows failures; (2) Clean up use of CDSConfig::is_xxx(); (3) Consolidate SpringPetClinic.java tests
7e207bd7: Merged ProblemList-premain.txt into ProblemList.txt
cb30dfbb: Renamed -XX:+ArchiveInvokeDynamic to -XX:+AOTInvokeDynamicLinking, and changed to a diagnostic flag
641fdabf: More clean up of -XX:+AOTInvokeDynamicLinking before upstreaming to mainline
c319a3ee: AOT G1 barrier loads region grain and card shifts via runtime constants data area
06f6f57b: Merge branch 'master' of https://github.com/openjdk/leyden into premain
2d15c193: temp work-around to get premain to work with JDK-8338532 and JDK-8336856; todo -- fix this more properly
126461c0: Enable 1-step workflow with Epsilon GC
a371fddb: Merge branch 'premain' of https://github.com/openjdk/leyden into premain
7f633b95: Keep cached strings longer. Fix ExternalsRecorder::print_statistics()
cc111760: Cleanup AOT runtime constants code
b96c5113: Fix various build configs for clean PRs and happy CIs
1bd62091: Simplified the archiving of enum class mirrors
3a84df9d: Renamed classPreinitializer.cpp to aotClassInitializer.cpp; more heapShared.cpp clean up
e33ecb3e: Simplified dumping of MethodType with the AOTHolder pattern
9bab6122: Experimental feature to allow special instrumentation code to be executed during training run. See CacheOnlyClassesIn.java for details
242e33d3: More clean up for upstreaming JDK-8329706 Implement -XX:+AOTClassLinking
cf036d70: Fix SCCache closing races
d7bd0ccc: Rework rich PrintCompilation logs
678b50d1: Prefer to take AOT method from the SC queue faster
5fd814c0: Improve incapsulation
d23b9f2d: Mark Leyden tests that require artifacts with "external-dep"
07ad2086: Tidy up
a8b8d7da: Reformat internal profiling stats
77811091: Method name AOTClassInitializer::has_non_default_static_fields  should be has_default_static_fields
6c83df44: 8340559: [premain] java process crash with -XX:+PrintTrainingInfo
7a6fadca: 8340869: [premain] NullPointerException with LambdaWithUseImplMethodHandle.java
ac3f40d0: 8340836: Method.invoke fails with java.lang.AssertionError with AOTClassLinking and system assertions enabled
21d3ed73: Merged some code from upstreaming PRs of JEP 483
9cf06243: Added some CDSHeapVerifier rules for ArchiveLoaderLookupCache
5bd0b880: Sync some files with latest JEP 483 PRs
173fb74c: 8339020: Remove unused HeapShared::calculate_oopmap
618907c4: 8339351: Remove duplicate line in FileMapHeader::print
e9521b21: 8338912: CDS: Segmented roots array
17571b6d: Merge branch 'master' into premain
cc1d328d: Merge branch 'master' into premain
ce1b43aa: Need to clean up _orig_to_scratch_object_table when classes are unloaded (e.g., when jdk.internal.event.Event gets rewritten by JFR)
a3a64c76: Merge branch 'master' into premain
7b72b048: FileMapInfo::write_bitmap_region bug after premain merge
b187389b: Merge branch 'master' into premain
ab9d91e5: Merge branch 'master' into premain
d504df0e: fix g1 barrier register use after late expand
5029a175: Ensure branch target is correctly relocated
88b6b4b2: further correction to register use in g1 barriers
4e0412e6: Enable 1-step workflow with Shenandoah GC
a1672ea6: Merge branch 'premain' of https://github.com/openjdk/leyden into premain
21640b10: Make TD::allocate() a template to reduce boilerplate
52eee916: fix -Xlog:init output
1c6e576c: Cleanup includes
cd8c4f40: Remove MethodInfoDictionary
8ebb8318: Merge branch 'master' into premain
163cae69: Fixed merge
f4e1f040: Merge branch jdk/master into premain
e4017de3: Prepare to merge with 41a2d49f0a1ed298b8ab023ce634335464454fe7 - hand merge some files
cb8c7904: Merge remote-tracking branch 'jdk/master' into premain
b5459e0a: Fixed merge
ac533a7e: Fixed CDSAccess::get_archived_object_permanent_index() to only keep alive archived oops that have a permanent index (previously every archived oop was kept alive)
c2871bb1: Merge branch 'premain' of https://github.com/openjdk/leyden into premain
b3e5a418: Remove unnecessary logs for CDSAccess::get_archived_object_permanent_index()

From ioi.lam at oracle.com  Fri Nov 22 05:27:10 2024
From: ioi.lam at oracle.com (ioi.lam at oracle.com)
Date: Thu, 21 Nov 2024 21:27:10 -0800
Subject: Temp branch for merging JEP 483 into leyden/premain
Message-ID: <e4acd156-4c16-4359-bc64-7b498c84d5fc@oracle.com>

As mentioned in today's Leyden meeting, I have merged JEP 483 into 
premain in my local repo. Since there are some new failures, I've put 
the merged version in a temporary branch:

https://github.com/openjdk/leyden/tree/premain-temp-merge-after-jep-483

[1] I have disabled some of the new CDS optimizations. See cdsConfig.cpp

 ??? // FIXME -- leyden+JEP483 merge {
 ??? FLAG_SET_ERGO(ArchiveDynamicProxies, false);
 ??? FLAG_SET_ERGO(ArchiveLoaderLookupCache, false);
 ??? FLAG_SET_ERGO(ArchivePackages, false);
 ??? FLAG_SET_ERGO(ArchiveProtectionDomains, false);
 ??? FLAG_SET_ERGO(ArchiveReflectionData, false);

The reason is that JEP 483 is more aggressive in picking classes to be 
AOT-initialized. For example, with the -XX:+ArchiveProtectionDomains 
option, after the 483 merge, the jdk.internal.loader.NativeLibraries 
class is AOT-initialized. As a result, the static field 
nativeLibraryLockMap is archived, so it remembers all the native libs 
that were loaded during the assembly phase. When we attempt to load 
these libs again during production, we get an "is being loaded in 
another classloader" error.

Therefore, we need more refactoring of the core libraries (such as 
adding runtimeSetup() methods) before some of these optimizations can be 
enabled again.

[2] There's a new failure in the AOT compiler. Could someone familiar 
with the compiler take a look?

#? Internal Error 
(/opt/mach5/mesos/work_dir/slaves/3fe4557b-066a-4392-9b05-ef874140c35c-S62/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/cfba4e4e-51a1-47f1-9d82-c6f09de94dd2/runs/e5810681-4ef9-46d2-9b14-5ef945fda550/workspace/open/src/hotspot/share/code/SCCache.cpp:4205), 
pid=1235, tid=1253
#? fatal error: Address 0x00007fe4817c5d30 for <unknown> is missing in 
SCA table

V? [libjvm.so+0x366795]? SCAddressTable::id_for_address(unsigned char*, 
RelocIterator, CodeBuffer*) [clone .part.0]+0x455 (SCCache.cpp:4205)
V? [libjvm.so+0x36df0a]? SCCache::write_relocations(CodeBuffer*, 
unsigned int&)+0x68a? (SCCache.cpp:4147)
V? [libjvm.so+0x3702d2]? SCCache::write_nmethod(methodHandle const&, 
int, int, CodeOffsets*, int, DebugInformationRecorder*, Dependencies*, 
CodeBuffer*, int, OopMapSet*, ExceptionHandlerTable*, 
ImplicitExceptionTable*, AbstractCompiler*, CompLevel, bool, bool, bool, 
bool, bool, bool)+0xb92? (SCCache.cpp:3335)
V? [libjvm.so+0x3707b2]? SCCache::store_nmethod(methodHandle const&, 
int, int, CodeOffsets*, int, DebugInformationRecorder*, Dependencies*, 
CodeBuffer*, int, OopMapSet*, ExceptionHandlerTable*, 
ImplicitExceptionTable*, AbstractCompiler*, CompLevel, bool, bool, bool, 
bool, bool, bool)+0x182? (SCCache.cpp:3102)
V? [libjvm.so+0x96ac99]? ciEnv::register_method(ciMethod*, int, 
CodeOffsets*, int, CodeBuffer*, int, OopMapSet*, ExceptionHandlerTable*, 
ImplicitExceptionTable*, AbstractCompiler*, bool, bool, bool, bool, 
bool, bool, int, bool, SCCEntry*)+0xcf9? (ciEnv.cpp:1099)
V? [libjvm.so+0x161c22e]? PhaseOutput::install_code(ciMethod*, int, 
AbstractCompiler*, bool, bool)+0x15e? (output.cpp:3445)
V? [libjvm.so+0xa9f46b]? Compile::Code_Gen()+0x5cb (compile.cpp:3036)
V? [libjvm.so+0xaa216f]? Compile::Compile(ciEnv*, ciMethod*, int, 
Options, DirectiveSet*)+0x1c7f? (compile.cpp:886)
V? [libjvm.so+0x8dfde1]? C2Compiler::compile_method(ciEnv*, ciMethod*, 
int, bool, DirectiveSet*)+0x261? (c2compiler.cpp:172)
V? [libjvm.so+0xaaff07] 
CompileBroker::invoke_compiler_on_method(CompileTask*)+0xcb7 
(compileBroker.cpp:2626)
V? [libjvm.so+0xab2448] CompileBroker::compiler_thread_loop()+0x758 
(compileBroker.cpp:2272)
V? [libjvm.so+0xfac4ce]? JavaThread::thread_main_inner()+0xee 
(javaThread.cpp:774)

On Linux/x64, this can be reliably reproduced with 
test/hotspot/jtreg/runtime/cds/appcds/applications/QuarkusGettingStarted.java#leyden

On aarch64 (Linux or MacOS), this failure seems to affect many more test 
cases. Also, instead of <unknown>, we see

#? Internal Error (SCCache.cpp:4159), pid=87789, tid=28419
#? fatal error: Address 0x0000000114d4aa80 for 
Stub:_large_arrays_hashcode_boolean is missing in SCA table

Thanks

- Ioi


From ioi.lam at oracle.com  Fri Nov 22 05:33:11 2024
From: ioi.lam at oracle.com (ioi.lam at oracle.com)
Date: Thu, 21 Nov 2024 21:33:11 -0800
Subject: [RFR] 8344140: Refactor the discovery of AOT cache artifacts
Message-ID: <6cc4d276-44d3-4dec-9376-3d4f44e31f64@oracle.com>

FYI - This is a PR for the JDK mainline related to Leyden development. 
Please comment on the PR page:

https://github.com/openjdk/jdk/pull/22291


====================
This is a clean up after the initial integration for JEP 483

+ Merged 3 loops into 1 for discovering the classes and oops that should
 ? be stored in the AOT cache. About 250 lines of code are deleted.

+ Added comments in aotArtifactFinder.hpp to describe the logic, which
 ? is much simpler than before.

+ Enum classes are no longer unconditionally AOT-initialized -- an Enum
 ? class is AOT-initialized only if we find an archived oop of this type.

Note to reviewers:

One consequence of this refactoring is that archived oops are now
discovered (by heapShared.cpp) before the metadata are copied (by 
ArchiveBuilder).
There are some functions that depend on the old order (metadata were copied
before oop discovery), so I had to shuffle them around. Examples are
Modules::check_archived_module_oop(), or the new code in 
Klass::remove_java_mirror().


From asmehra at redhat.com  Fri Nov 22 17:06:41 2024
From: asmehra at redhat.com (Ashutosh Mehra)
Date: Fri, 22 Nov 2024 12:06:41 -0500
Subject: Temp branch for merging JEP 483 into leyden/premain
In-Reply-To: <e4acd156-4c16-4359-bc64-7b498c84d5fc@oracle.com>
References: <e4acd156-4c16-4359-bc64-7b498c84d5fc@oracle.com>
Message-ID: <CAKt0pySzZ2X0EQ4bC4Lp6n3bB2zkX4uuEv0AS3PYF7FQkCCUmw@mail.gmail.com>

>
> #  Internal Error
>
> (/opt/mach5/mesos/work_dir/slaves/3fe4557b-066a-4392-9b05-ef874140c35c-S62/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/cfba4e4e-51a1-47f1-9d82-c6f09de94dd2/runs/e5810681-4ef9-46d2-9b14-5ef945fda550/workspace/open/src/hotspot/share/code/SCCache.cpp:4205),
> pid=1235, tid=1253
> #  fatal error: Address 0x00007fe4817c5d30 for <unknown> is missing in
> SCA table


This error on Linux/x64 can be fixed by increasing MAX_STR_COUNT from 200
to 300.
Here is the patch for that:

diff --git a/src/hotspot/share/code/SCCache.cpp
b/src/hotspot/share/code/SCCache.cpp
index f42fab991a5..5df7a84f5d6 100644
--- a/src/hotspot/share/code/SCCache.cpp
+++ b/src/hotspot/share/code/SCCache.cpp
@@ -3956,7 +3956,7 @@ SCAddressTable::~SCAddressTable() {
   }
 }

-#define MAX_STR_COUNT 200
+#define MAX_STR_COUNT 300
 static const char* _C_strings[MAX_STR_COUNT] = {nullptr};
 static int _C_strings_count = 0;
 static int _C_strings_s[MAX_STR_COUNT] = {0};

 With this change
test/hotspot/jtreg/runtime/cds/appcds/applications/QuarkusGettingStarted.java#leyden
works fine.

On aarch64 (Linux or MacOS), this failure seems to affect many more test
> cases. Also, instead of <unknown>, we see
>
> #  Internal Error (SCCache.cpp:4159), pid=87789, tid=28419
> #  fatal error: Address 0x0000000114d4aa80 for
> Stub:_large_arrays_hashcode_boolean is missing in SCA table


I don't have an aarch64 machine, but the error message indicates some
stub(s) in cpu/aarch64/stubRoutines_aarch64.hpp are missing in the SCA
table.
I think adding them should resolve this issue. Patch for adding those stubs
is:

diff --git a/src/hotspot/share/code/SCCache.cpp
b/src/hotspot/share/code/SCCache.cpp
index f42fab991a5..89b5a8bf0e2 100644
--- a/src/hotspot/share/code/SCCache.cpp
+++ b/src/hotspot/share/code/SCCache.cpp
@@ -3813,6 +3813,11 @@ void SCAddressTable::init() {
   SET_ADDRESS(_stubs, StubRoutines::aarch64::count_positives());
   SET_ADDRESS(_stubs, StubRoutines::aarch64::count_positives_long());
   SET_ADDRESS(_stubs, StubRoutines::aarch64::large_array_equals());
+  SET_ADDRESS(_stubs,
StubRoutines::aarch64::large_arrays_hashcode(T_BOOLEAN));
+  SET_ADDRESS(_stubs,
StubRoutines::aarch64::large_arrays_hashcode(T_BYTE));
+  SET_ADDRESS(_stubs,
StubRoutines::aarch64::large_arrays_hashcode(T_CHAR));
+  SET_ADDRESS(_stubs, StubRoutines::aarch64::large_arrays_hashcode(T_INT));
+  SET_ADDRESS(_stubs,
StubRoutines::aarch64::large_arrays_hashcode(T_SHORT));
   SET_ADDRESS(_stubs, StubRoutines::aarch64::compare_long_string_LL());
   SET_ADDRESS(_stubs, StubRoutines::aarch64::compare_long_string_UU());
   SET_ADDRESS(_stubs, StubRoutines::aarch64::compare_long_string_LU());

@Ioi, can you please try these patches.

Thanks,
- Ashutosh Mehra


On Fri, Nov 22, 2024 at 12:27?AM <ioi.lam at oracle.com> wrote:

> As mentioned in today's Leyden meeting, I have merged JEP 483 into
> premain in my local repo. Since there are some new failures, I've put
> the merged version in a temporary branch:
>
> https://github.com/openjdk/leyden/tree/premain-temp-merge-after-jep-483
>
> [1] I have disabled some of the new CDS optimizations. See cdsConfig.cpp
>
>      // FIXME -- leyden+JEP483 merge {
>      FLAG_SET_ERGO(ArchiveDynamicProxies, false);
>      FLAG_SET_ERGO(ArchiveLoaderLookupCache, false);
>      FLAG_SET_ERGO(ArchivePackages, false);
>      FLAG_SET_ERGO(ArchiveProtectionDomains, false);
>      FLAG_SET_ERGO(ArchiveReflectionData, false);
>
> The reason is that JEP 483 is more aggressive in picking classes to be
> AOT-initialized. For example, with the -XX:+ArchiveProtectionDomains
> option, after the 483 merge, the jdk.internal.loader.NativeLibraries
> class is AOT-initialized. As a result, the static field
> nativeLibraryLockMap is archived, so it remembers all the native libs
> that were loaded during the assembly phase. When we attempt to load
> these libs again during production, we get an "is being loaded in
> another classloader" error.
>
> Therefore, we need more refactoring of the core libraries (such as
> adding runtimeSetup() methods) before some of these optimizations can be
> enabled again.
>
> [2] There's a new failure in the AOT compiler. Could someone familiar
> with the compiler take a look?
>
> #  Internal Error
> (/opt/mach5/mesos/work_dir/slaves/3fe4557b-066a-4392-9b05-ef874140c35c-S62/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/cfba4e4e-51a1-47f1-9d82-c6f09de94dd2/runs/e5810681-4ef9-46d2-9b14-5ef945fda550/workspace/open/src/hotspot/share/code/SCCache.cpp:4205),
>
> pid=1235, tid=1253
> #  fatal error: Address 0x00007fe4817c5d30 for <unknown> is missing in
> SCA table
>
> V  [libjvm.so+0x366795]  SCAddressTable::id_for_address(unsigned char*,
> RelocIterator, CodeBuffer*) [clone .part.0]+0x455 (SCCache.cpp:4205)
> V  [libjvm.so+0x36df0a]  SCCache::write_relocations(CodeBuffer*,
> unsigned int&)+0x68a  (SCCache.cpp:4147)
> V  [libjvm.so+0x3702d2]  SCCache::write_nmethod(methodHandle const&,
> int, int, CodeOffsets*, int, DebugInformationRecorder*, Dependencies*,
> CodeBuffer*, int, OopMapSet*, ExceptionHandlerTable*,
> ImplicitExceptionTable*, AbstractCompiler*, CompLevel, bool, bool, bool,
> bool, bool, bool)+0xb92  (SCCache.cpp:3335)
> V  [libjvm.so+0x3707b2]  SCCache::store_nmethod(methodHandle const&,
> int, int, CodeOffsets*, int, DebugInformationRecorder*, Dependencies*,
> CodeBuffer*, int, OopMapSet*, ExceptionHandlerTable*,
> ImplicitExceptionTable*, AbstractCompiler*, CompLevel, bool, bool, bool,
> bool, bool, bool)+0x182  (SCCache.cpp:3102)
> V  [libjvm.so+0x96ac99]  ciEnv::register_method(ciMethod*, int,
> CodeOffsets*, int, CodeBuffer*, int, OopMapSet*, ExceptionHandlerTable*,
> ImplicitExceptionTable*, AbstractCompiler*, bool, bool, bool, bool,
> bool, bool, int, bool, SCCEntry*)+0xcf9  (ciEnv.cpp:1099)
> V  [libjvm.so+0x161c22e]  PhaseOutput::install_code(ciMethod*, int,
> AbstractCompiler*, bool, bool)+0x15e  (output.cpp:3445)
> V  [libjvm.so+0xa9f46b]  Compile::Code_Gen()+0x5cb (compile.cpp:3036)
> V  [libjvm.so+0xaa216f]  Compile::Compile(ciEnv*, ciMethod*, int,
> Options, DirectiveSet*)+0x1c7f  (compile.cpp:886)
> V  [libjvm.so+0x8dfde1]  C2Compiler::compile_method(ciEnv*, ciMethod*,
> int, bool, DirectiveSet*)+0x261  (c2compiler.cpp:172)
> V  [libjvm.so+0xaaff07]
> CompileBroker::invoke_compiler_on_method(CompileTask*)+0xcb7
> (compileBroker.cpp:2626)
> V  [libjvm.so+0xab2448] CompileBroker::compiler_thread_loop()+0x758
> (compileBroker.cpp:2272)
> V  [libjvm.so+0xfac4ce]  JavaThread::thread_main_inner()+0xee
> (javaThread.cpp:774)
>
> On Linux/x64, this can be reliably reproduced with
>
> test/hotspot/jtreg/runtime/cds/appcds/applications/QuarkusGettingStarted.java#leyden
>
> On aarch64 (Linux or MacOS), this failure seems to affect many more test
> cases. Also, instead of <unknown>, we see
>
> #  Internal Error (SCCache.cpp:4159), pid=87789, tid=28419
> #  fatal error: Address 0x0000000114d4aa80 for
> Stub:_large_arrays_hashcode_boolean is missing in SCA table
>
> Thanks
>
> - Ioi
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/leyden-dev/attachments/20241122/a05d1da8/attachment.htm>

From macarte at openjdk.org  Fri Nov 22 19:27:16 2024
From: macarte at openjdk.org (Mat Carter)
Date: Fri, 22 Nov 2024 19:27:16 GMT
Subject: RFR: 8335358: [premain] Explore alternative ways to trigger the
 end of training run [v17]
In-Reply-To: <RqVNoxdgqloUtbXxgiiqePA6XRrHkuolETkbzmzr7r4=.2b32af0e-69fa-48cf-89aa-322371ef9fb2@github.com>
References: <RqVNoxdgqloUtbXxgiiqePA6XRrHkuolETkbzmzr7r4=.2b32af0e-69fa-48cf-89aa-322371ef9fb2@github.com>
Message-ID: <xeHdRK5CLBlEypO1SvbQgn8lDpvZ8JotCce2iOn1egY=.850ac6d4-3c6d-4ece-95c8-514209a2be30@github.com>

> AOT training can be ended using either
> 
> - -XX: AOTEndTrainingOnMethodEntry =Hello.someMethod [same syntax as CompileOnly]
> - -XX: AOTEndTrainingOnMethodEntry =Hello.someMethod,Hello.someOtherMethod,count=42
> - jcmd <pid> AOT.end_training
> 
> Also added programmatic hooks to System class:
> - bool System.AOTIsTraining()
> - void System.AOTEndTraining()
> 
> supports arm64 and x64
> 
> note: the AOTEndTrainingOnMethodEntry is ignored when not recording
> 
> JBS Issue: https://bugs.openjdk.org/browse/JDK-8335358

Mat Carter has updated the pull request incrementally with one additional commit since the last revision:

  Added programmatic AOT(Is|End)Training calls to System class

-------------

Changes:
  - all: https://git.openjdk.org/leyden/pull/21/files
  - new: https://git.openjdk.org/leyden/pull/21/files/88200565..62afc306

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=leyden&pr=21&range=16
 - incr: https://webrevs.openjdk.org/?repo=leyden&pr=21&range=15-16

  Stats: 57 lines in 6 files changed: 56 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/leyden/pull/21.diff
  Fetch: git fetch https://git.openjdk.org/leyden.git pull/21/head:pull/21

PR: https://git.openjdk.org/leyden/pull/21

From macarte at openjdk.org  Fri Nov 22 19:27:16 2024
From: macarte at openjdk.org (Mat Carter)
Date: Fri, 22 Nov 2024 19:27:16 GMT
Subject: RFR: 8335358: [premain] Explore alternative ways to trigger the
 end of training run [v15]
In-Reply-To: <kioi4yYILjx34ujFCGPxNw2VUe0FPXCHvCnMySbHT5k=.41d73d75-9ecb-4286-8476-f55b84fd8573@github.com>
References: <RqVNoxdgqloUtbXxgiiqePA6XRrHkuolETkbzmzr7r4=.2b32af0e-69fa-48cf-89aa-322371ef9fb2@github.com>
 <buyr9_DE8yvXU_J70o-qkGFdx-wVRKelPZ_O1zEwZPM=.ff1f3fcb-d15c-4c88-8402-2a5f5bb7fe56@github.com>
 <ZWA7GABBtU6KOk5Qv_87wtH_V6dua9FnRvjY4M8SJr4=.a8f9a304-7e98-486e-bf50-9e8fcaf5414d@github.com>
 <kioi4yYILjx34ujFCGPxNw2VUe0FPXCHvCnMySbHT5k=.41d73d75-9ecb-4286-8476-f55b84fd8573@github.com>
Message-ID: <NVRoy-Y1MdLiYrW47AqRy9wazlZSwuJESm_5HZkbLKE=.ccabc905-cf37-403f-83f3-23464b2811d5@github.com>

On Thu, 21 Nov 2024 17:36:20 GMT, Ioi Lam <iklam at openjdk.org> wrote:

> > Nice work, Mat!
> > Speaking of proposed API, there's a lot of complexity around `-XX:AOTEndTrainingOnMethodEntry=`. What's the primary use case for it? Does it add much for uncooperative case (no application changes) compared to `jcmd`?
> > For cooperative case, a dedicated method in JDK an application can call seems more appropriate.
> 
> I think this PR has its benefits:
> 
> * JCMD triggers are imprecise. If you using jcmd to trigger the end of training based on time, or based on parsing the app's log, it may capture too much or too little profile.
> * An API would require modifying the app.
> 
> So this PR allows precise timing (for apps that have a well-known Java method that's called during important state transition) without modifying the app itself.

Following our premain discussion I've added two methods to the System class.  AOTEndTraining() as discussed as well as AOTIsTraining() which may also be of use for those willing to modify their application.  As with the -XX:EndTrainingOnMethodEntry we'll review workflows in premain and settle on an approach before any commit to tip

-------------

PR Comment: https://git.openjdk.org/leyden/pull/21#issuecomment-2494626228

From iklam at openjdk.org  Fri Nov 22 20:39:41 2024
From: iklam at openjdk.org (Ioi Lam)
Date: Fri, 22 Nov 2024 20:39:41 GMT
Subject: RFR: 8335358: [premain] Explore alternative ways to trigger the
 end of training run [v17]
In-Reply-To: <xeHdRK5CLBlEypO1SvbQgn8lDpvZ8JotCce2iOn1egY=.850ac6d4-3c6d-4ece-95c8-514209a2be30@github.com>
References: <RqVNoxdgqloUtbXxgiiqePA6XRrHkuolETkbzmzr7r4=.2b32af0e-69fa-48cf-89aa-322371ef9fb2@github.com>
 <xeHdRK5CLBlEypO1SvbQgn8lDpvZ8JotCce2iOn1egY=.850ac6d4-3c6d-4ece-95c8-514209a2be30@github.com>
Message-ID: <Az6w78K6kh2cVHmfKTxZf4MSSO-wfREFQ6uqGNYaOM8=.94b4c1ed-2839-4b60-9d24-e660148cfb56@github.com>

On Fri, 22 Nov 2024 19:27:16 GMT, Mat Carter <macarte at openjdk.org> wrote:

>> AOT training can be ended using either
>> 
>> - -XX: AOTEndTrainingOnMethodEntry =Hello.someMethod [same syntax as CompileOnly]
>> - -XX: AOTEndTrainingOnMethodEntry =Hello.someMethod,Hello.someOtherMethod,count=42
>> - jcmd <pid> AOT.end_training
>> 
>> Also added programmatic hooks to System class:
>> - bool System.AOTIsTraining()
>> - void System.AOTEndTraining()
>> 
>> supports arm64 and x64
>> 
>> note: the AOTEndTrainingOnMethodEntry is ignored when not recording
>> 
>> JBS Issue: https://bugs.openjdk.org/browse/JDK-8335358
>
> Mat Carter has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Added programmatic AOT(Is|End)Training calls to System class

LGTM. Mat, if are you ready to integrate, you can enter the /integrate command and I will sponsor it.

-------------

Marked as reviewed by iklam (Committer).

PR Review: https://git.openjdk.org/leyden/pull/21#pullrequestreview-2455593713

From duke at openjdk.org  Fri Nov 22 21:50:33 2024
From: duke at openjdk.org (duke)
Date: Fri, 22 Nov 2024 21:50:33 GMT
Subject: RFR: 8335358: [premain] Explore alternative ways to trigger the
 end of training run [v17]
In-Reply-To: <xeHdRK5CLBlEypO1SvbQgn8lDpvZ8JotCce2iOn1egY=.850ac6d4-3c6d-4ece-95c8-514209a2be30@github.com>
References: <RqVNoxdgqloUtbXxgiiqePA6XRrHkuolETkbzmzr7r4=.2b32af0e-69fa-48cf-89aa-322371ef9fb2@github.com>
 <xeHdRK5CLBlEypO1SvbQgn8lDpvZ8JotCce2iOn1egY=.850ac6d4-3c6d-4ece-95c8-514209a2be30@github.com>
Message-ID: <lrjzN_q4Ss8SN7QRV8kJypnpMQfnR5XcS49Fs9JA5ow=.7e4baf61-92db-4e5c-82b0-459de9931bfd@github.com>

On Fri, 22 Nov 2024 19:27:16 GMT, Mat Carter <macarte at openjdk.org> wrote:

>> AOT training can be ended using either
>> 
>> - -XX: AOTEndTrainingOnMethodEntry =Hello.someMethod [same syntax as CompileOnly]
>> - -XX: AOTEndTrainingOnMethodEntry =Hello.someMethod,Hello.someOtherMethod,count=42
>> - jcmd <pid> AOT.end_training
>> 
>> Also added programmatic hooks to System class:
>> - bool System.AOTIsTraining()
>> - void System.AOTEndTraining()
>> 
>> supports arm64 and x64
>> 
>> note: the AOTEndTrainingOnMethodEntry is ignored when not recording
>> 
>> JBS Issue: https://bugs.openjdk.org/browse/JDK-8335358
>
> Mat Carter has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Added programmatic AOT(Is|End)Training calls to System class

@macarte 
Your change (at version 62afc30608a81afdb4067a7ece8841329b01b2f8) is now ready to be sponsored by a Committer.

-------------

PR Comment: https://git.openjdk.org/leyden/pull/21#issuecomment-2494919970

From duke at openjdk.org  Sat Nov 23 00:01:10 2024
From: duke at openjdk.org (duke)
Date: Sat, 23 Nov 2024 00:01:10 GMT
Subject: git: openjdk/leyden: premain-temp-merge-after-jep-483: Fix SCCache
 crashes
Message-ID: <2fc1db05-9ae6-4cf9-99bb-b0879520362f@openjdk.org>

Changeset: ff46e8e6
Branch: premain-temp-merge-after-jep-483
Author:    Vladimir Ivanov <vladimir.x.ivanov at oracle.com>
Date:      2024-11-22 16:00:25 +0000
URL:       https://git.openjdk.org/leyden/commit/ff46e8e6a4c2a1121a17b42a1e8764e457ae0c92

Fix SCCache crashes

! src/hotspot/share/code/SCCache.cpp


From iklam at openjdk.org  Sat Nov 23 00:24:09 2024
From: iklam at openjdk.org (Ioi Lam)
Date: Sat, 23 Nov 2024 00:24:09 GMT
Subject: git: openjdk/leyden: premain: 8335358: [premain] Explore alternative
 ways to trigger the end of training run
Message-ID: <4ca66f59-9331-4841-86c8-03477f3c40bd@openjdk.org>

Changeset: 6e6030b9
Branch: premain
Author:    Mat Carter <macarte at openjdk.org>
Committer: Ioi Lam <iklam at openjdk.org>
Date:      2024-11-23 00:23:36 +0000
URL:       https://git.openjdk.org/leyden/commit/6e6030b917f1dfcd4ea7c6207ce0626bf5452c3f

8335358: [premain] Explore alternative ways to trigger the end of training run

Reviewed-by: iklam

! README.md
! src/hotspot/cpu/aarch64/interp_masm_aarch64.cpp
! src/hotspot/cpu/aarch64/interp_masm_aarch64.hpp
! src/hotspot/cpu/aarch64/templateInterpreterGenerator_aarch64.cpp
! src/hotspot/cpu/arm/templateInterpreterGenerator_arm.cpp
! src/hotspot/cpu/ppc/templateInterpreterGenerator_ppc.cpp
! src/hotspot/cpu/riscv/templateInterpreterGenerator_riscv.cpp
! src/hotspot/cpu/s390/templateInterpreterGenerator_s390.cpp
! src/hotspot/cpu/x86/interp_masm_x86.cpp
! src/hotspot/cpu/x86/interp_masm_x86.hpp
! src/hotspot/cpu/x86/templateInterpreterGenerator_x86.cpp
! src/hotspot/share/c1/c1_GraphBuilder.cpp
! src/hotspot/share/c1/c1_LIRGenerator.cpp
! src/hotspot/share/c1/c1_Runtime1.cpp
! src/hotspot/share/cds/cdsConfig.cpp
! src/hotspot/share/cds/cdsConfig.hpp
+ src/hotspot/share/cds/cdsEndTrainingUpcall.cpp
+ src/hotspot/share/cds/cdsEndTrainingUpcall.hpp
! src/hotspot/share/cds/cds_globals.hpp
! src/hotspot/share/cds/metaspaceShared.cpp
! src/hotspot/share/cds/metaspaceShared.hpp
! src/hotspot/share/ci/ciMethod.hpp
! src/hotspot/share/compiler/methodMatcher.cpp
! src/hotspot/share/compiler/methodMatcher.hpp
! src/hotspot/share/include/jvm.h
! src/hotspot/share/interpreter/abstractInterpreter.cpp
! src/hotspot/share/interpreter/abstractInterpreter.hpp
! src/hotspot/share/interpreter/templateInterpreterGenerator.cpp
! src/hotspot/share/interpreter/templateInterpreterGenerator.hpp
! src/hotspot/share/interpreter/zero/zeroInterpreterGenerator.cpp
! src/hotspot/share/interpreter/zero/zeroInterpreterGenerator.hpp
! src/hotspot/share/oops/instanceKlass.cpp
! src/hotspot/share/oops/method.cpp
! src/hotspot/share/oops/methodFlags.hpp
! src/hotspot/share/opto/graphKit.hpp
! src/hotspot/share/opto/parse1.cpp
! src/hotspot/share/opto/parseHelper.cpp
! src/hotspot/share/opto/runtime.cpp
! src/hotspot/share/opto/runtime.hpp
! src/hotspot/share/prims/jvm.cpp
! src/hotspot/share/runtime/globals.hpp
! src/hotspot/share/runtime/init.cpp
+ src/hotspot/share/runtime/methodDetails.cpp
+ src/hotspot/share/runtime/methodDetails.hpp
+ src/hotspot/share/runtime/runtimeUpcallNop.cpp
+ src/hotspot/share/runtime/runtimeUpcallNop.hpp
+ src/hotspot/share/runtime/runtimeUpcalls.cpp
+ src/hotspot/share/runtime/runtimeUpcalls.hpp
! src/hotspot/share/services/diagnosticCommand.cpp
! src/hotspot/share/services/diagnosticCommand.hpp
! src/java.base/share/classes/java/lang/System.java
! src/java.base/share/native/libjava/System.c


From macarte at openjdk.org  Sat Nov 23 00:26:33 2024
From: macarte at openjdk.org (Mat Carter)
Date: Sat, 23 Nov 2024 00:26:33 GMT
Subject: Integrated: 8335358: [premain] Explore alternative ways to trigger the
 end of training run
In-Reply-To: <RqVNoxdgqloUtbXxgiiqePA6XRrHkuolETkbzmzr7r4=.2b32af0e-69fa-48cf-89aa-322371ef9fb2@github.com>
References: <RqVNoxdgqloUtbXxgiiqePA6XRrHkuolETkbzmzr7r4=.2b32af0e-69fa-48cf-89aa-322371ef9fb2@github.com>
Message-ID: <xHL4c76QaWAXxsC6DA4GoQuX59FYlvmRJdUKg1Oq3_Y=.742632cf-7539-452c-9f79-0a366d3605ae@github.com>

On Wed, 11 Sep 2024 21:34:57 GMT, Mat Carter <macarte at openjdk.org> wrote:

> AOT training can be ended using either
> 
> - -XX: AOTEndTrainingOnMethodEntry =Hello.someMethod [same syntax as CompileOnly]
> - -XX: AOTEndTrainingOnMethodEntry =Hello.someMethod,Hello.someOtherMethod,count=42
> - jcmd <pid> AOT.end_training
> 
> Also added programmatic hooks to System class:
> - bool System.AOTIsTraining()
> - void System.AOTEndTraining()
> 
> supports arm64 and x64
> 
> note: the AOTEndTrainingOnMethodEntry is ignored when not recording
> 
> JBS Issue: https://bugs.openjdk.org/browse/JDK-8335358

This pull request has now been integrated.

Changeset: 6e6030b9
Author:    Mat Carter <macarte at openjdk.org>
Committer: Ioi Lam <iklam at openjdk.org>
URL:       https://git.openjdk.org/leyden/commit/6e6030b917f1dfcd4ea7c6207ce0626bf5452c3f
Stats:     1254 lines in 52 files changed: 1196 ins; 10 del; 48 mod

8335358: [premain] Explore alternative ways to trigger the end of training run

Reviewed-by: iklam

-------------

PR: https://git.openjdk.org/leyden/pull/21

From Matthew.Carter at microsoft.com  Tue Nov 26 19:47:35 2024
From: Matthew.Carter at microsoft.com (Mat Carter)
Date: Tue, 26 Nov 2024 19:47:35 +0000
Subject: Premain options for triggering end of training
Message-ID: <87AD71EC-3FE9-4C18-BF50-02BDABC59081@microsoft.com>

The following three options have been integrated into premain to assist with the recording of
training data.  What parts make it to tip (and in what format) will be based on feedback from
 training workflows as developers explore Leyden.

Command line option: -XX:AOTEndTrainingOnMethodEntry=[methods][,count]
- methods: comma separated list of methods specified in a similar manner to -XX:CompileOnly
- count: trigger end of training after ?count? tracked methods invoked (total, not per method)

JCmd option:  jcmd AOT.end_training
- ends training

API option (don?t panic: System was simply the easiest point to integrate support for
experimentation):

void System.AOTEndTraining()
bool System.AOTIsTraining()
- returns true if a training run and the recording of training data has not yet stopped

Ending training does not terminate the application; it simply stops recording and dumps
out the data for creating the AOT cache.

If none of the methods are used (or the ?count? was not met) then the application will stop
recording data when the application terminates.

This work is based on the enhancement request in JBS [1]; the PR is [2] and commit is [3]

Cheers
Mat

[1] https://bugs.openjdk.org/browse/JDK-8335358
[2] https://github.com/openjdk/leyden/pull/21#issuecomment-2495140178
[3] https://github.com/openjdk/leyden/commit/6e6030b917f1dfcd4ea7c6207ce0626bf5452c3f
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: Message signed with OpenPGP
URL: <https://mail.openjdk.org/pipermail/leyden-dev/attachments/20241126/9647b5c5/signature.asc>

From duke at openjdk.org  Wed Nov 27 22:21:08 2024
From: duke at openjdk.org (duke)
Date: Wed, 27 Nov 2024 22:21:08 GMT
Subject: git: openjdk/leyden: premain-temp-merge-after-jep-483: Don't allocate
 new C strings on the fly
Message-ID: <a3fdf07c-a612-4365-b5ed-6341b7e73531@openjdk.org>

Changeset: 58565167
Branch: premain-temp-merge-after-jep-483
Author:    Vladimir Ivanov <vladimir.x.ivanov at oracle.com>
Date:      2024-11-27 14:20:00 +0000
URL:       https://git.openjdk.org/leyden/commit/58565167d5c8ba59d3034cf2ce4c249e4d7a7d7a

Don't allocate new C strings on the fly

! src/hotspot/cpu/x86/methodHandles_x86.cpp


From duke at openjdk.org  Wed Nov 27 22:24:37 2024
From: duke at openjdk.org (duke)
Date: Wed, 27 Nov 2024 22:24:37 GMT
Subject: git: openjdk/leyden: premain-temp-merge-after-jep-483: fix build
Message-ID: <9a292cc3-2645-46af-8bcf-dbc2908892cc@openjdk.org>

Changeset: 6ca0c9ef
Branch: premain-temp-merge-after-jep-483
Author:    Vladimir Ivanov <vladimir.x.ivanov at oracle.com>
Date:      2024-11-27 14:23:55 +0000
URL:       https://git.openjdk.org/leyden/commit/6ca0c9efcbe3f84a9c1f77be670d485e55945e82

fix build

! src/hotspot/cpu/x86/methodHandles_x86.cpp


From duke at openjdk.org  Wed Nov 27 22:26:46 2024
From: duke at openjdk.org (duke)
Date: Wed, 27 Nov 2024 22:26:46 GMT
Subject: git: openjdk/leyden: premain-temp-merge-after-jep-483: fix build
Message-ID: <a202b777-a05d-4b8b-b7b7-dee16fe406aa@openjdk.org>

Changeset: ba71a22b
Branch: premain-temp-merge-after-jep-483
Author:    Vladimir Ivanov <vladimir.x.ivanov at oracle.com>
Date:      2024-11-27 14:24:34 +0000
URL:       https://git.openjdk.org/leyden/commit/ba71a22b7764f65942fcaa4e59b0e98e0e791435

fix build

! src/hotspot/cpu/x86/methodHandles_x86.cpp