RFR: 8348426: Generate binary file for -XX:AOTMode=record -XX:AOTConfiguration=file [v8]

[John R Rose]

On Tue, 25 Feb 2025 01:11:25 GMT, Ioi Lam <iklam at openjdk.org> wrote:

>> Currently, with `java -XX:AOTMode=record -XX:AOTConfiguration=file ...`, a text file is written. The file contains the names of loaded classes, indices of resolved constant pools entries, etc, that are easily represented in text.
>> 
>> With the upcoming 2nd JEP of the Leyden project, [JDK-8325147](https://bugs.openjdk.org/browse/JDK-8325147) (Ahead-of-Time Method Profiling), the AOT config file needs to record complex data structures that are difficult to represent in text (we would need code for serializing hierarchical data structures to/from text). Also, a next step after [JDK-8325147](https://bugs.openjdk.org/browse/JDK-8325147) would be to support hidden classes that have no predictable names. Representing such classes with textual names would become another challenge.
>> 
>> To prepare for  [JDK-8325147](https://bugs.openjdk.org/browse/JDK-8325147), this PR writes the AOT configuration file in a **binary format** (essentially the same format as a CDS archive file). This allows arbitrary data associated with the cached classes to be processed and stored using the existing `MetaspaceClosure` API (which can recursively copy C++ objects). Such a change in the file format is allowed by [JEP 483](https://openjdk.org/jeps/483):
>> 
>>> the format of the configuration and cache files is not specified and is subject to change without notice.
>> 
>> **Notes for reviewers:**
>> 
>> - Although the new config file format is essentially the same as a CDS "static" archive, for sanity, we use a different magic number so that the config file cannot be accidentally used as a CDS archive. See new tests inside AOTFlags.java.
>> - After this PR, the CDS "static" archive can be dumped in three modes: "classic", "preimage", and "final". See new comments in cdsConfig.hpp.
>> - The main starting point of this PR is `CDSConfig::check_aot_flags()` - it checks the existence of `-XX:AOTConfiguration` and `-XX:AOTMode` to configure the JVM to dump the CDS "preimage" or "final" archives as necessary.
>> - Most of the other changes are checks for `CDSConfig::is_dumping_preimage_static_archive()` and `CDSConfig::is_dumping_final_static_archive()` to handle subtlle differences between the different dumping modes.
>> - I also updated the UL messages to use the new JEP 483 terminology ("AOT cache", "AOT configuration file",  etc) when JEP 483 options are specified.
>> 
>> **Misc Note**
>> - The changes in [CDS.java and RunTests.gmk](https://github.com/iklam/jdk/commit/0e77a35c25a968c7d931931bc108ccb...
>
> Ioi Lam has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 15 commits:
> 
>  - all tests in runtime/cds/appcds/aotClassLinking should be excluded for hotspot_appcds_dynamic testing
>  - @ashu-mehra comment - simplified _archived_cpp_vtptrs; also fixed old comments near by
>  - Merge branch 'master' into 8348426-binary-aot-config-file
>  - Merge branch 'master' into 8348426-binary-aot-config-file
>  - @ashu-mehra comments
>  - @calvinccheung comments
>  - Improved JTREG_AOT_JDK=true so we do not need to add test code into the JDK itself
>  - Improve error message when AOTMode=create has an incompatible classpath
>  - Fixed test cases @vnkozlov
>  - Update "make test JTREG_AOT_JDK=true ..." to work with binary AOT configuration
>  - ... and 5 more: https://git.openjdk.org/jdk/compare/990d40e9...1ec67c11

Terminology suggestion:  "preimage" has a math meaning which is a little bit distinct from the usage here.  A good software example of a math preimage would be a data structure you are going to copy into another container, with some kind of transform. 

Meanwhile, "aotconfig" is a very specific term (also visible to the user) which means exactly the contents of the file being change.

I suggest changing "preimage" in this patch to "aotconfig", more or less uniformly.  Then it will be really clear that we are talking about the `foo.aotconfig` file in the JVM source code.

Indeed, the word "preimage" is correct (in the math sense) at present when we copy metadata loaded into the training run, into the atoconfig file, all the way through the assembly phase to the AOT cache.  But that is kind of accidental; the assembly phase might choose to reload classes from scratch some day.  And we certainly rebuild AOT code (SCC) from scratch now.   The necessary role of the aotconfig is to provide configuration data that drives AOT asset creation; it is sort of accidental when the aotconfig provides literal AOT assets to copy through.

I tipped over into this comment when I realized that "aotconfig file" is the same as "preinage file".

-------------

PR Comment: https://git.openjdk.org/jdk/pull/23484#issuecomment-2688519203