AOT Map changes during production run
Ashutosh Mehra
asmehra at redhat.com
Thu Oct 9 19:25:10 UTC 2025
I am able to reproduce the problem Maria is seeing.
Digging deeper it looks like when the map is generated during the
production run, the VM skips classes loaded by the custom loaders.
VM gets the list of classes using
SystemDictionaryShared::get_all_archived_classes
<https://github.com/openjdk/jdk/blob/1992b69a4794d1f2f65eaeb6dbb1e1e23a948b6e/src/hotspot/share/classfile/systemDictionaryShared.cpp#L1419-L1423>
which only iterates through the classes loaded by built-in loaders.
With the following patch, I am able to get all the classes in the map file
generated in the prod run:
diff --git a/src/hotspot/share/classfile/systemDictionaryShared.cpp
b/src/hotspot/share/classfile/systemDictionaryShared.cpp
index b092e71f4e7..7e71dbefbfb 100644
--- a/src/hotspot/share/classfile/systemDictionaryShared.cpp
+++ b/src/hotspot/share/classfile/systemDictionaryShared.cpp
@@ -1420,6 +1420,9 @@ void
SystemDictionaryShared::get_all_archived_classes(bool is_static_archive, Gr
get_archive(is_static_archive)->_builtin_dictionary.iterate([&] (const
RunTimeClassInfo* record) {
classes->append(record->klass());
});
+ get_archive(is_static_archive)->_unregistered_dictionary.iterate([&]
(const RunTimeClassInfo* record) {
+ classes->append(record->klass());
+ });
}
class SharedDictionaryPrinter : StackObj {
@Ioi Lam <ioi.lam at oracle.com> Do you think we should do this?
Thanks,
- Ashutosh Mehra
On Thu, Oct 9, 2025 at 1:05 PM María Arias de Reyna Dominguez <
mariasde at redhat.com> wrote:
> Hi!
>
> I am using Infinispan to test this. So I edit their bin/server.conf file
> to add my own java arguments. Nothing magical beyond that.
>
> Depending on if the cache file already exists, I create or not the cache.
> And then I (used to) always generate the aot.map file:
>
> if [ -f "/home/delawen/infinispan-server-15.2.5.Final/aot.map" ]; then
> JAVA_OPTS="$JAVA_OPTS -XX:AOTCache=app.aot"
> else
> JAVA_OPTS="$JAVA_OPTS -XX:AOTCacheOutput=app.aot"
> fi
> JAVA_OPTS="$JAVA_OPTS -Xlog:aot+map=trace:file=aot.map:none:filesize=0"
>
> According to https://bugs.openjdk.org/browse/JDK-8362566 that should
> re-create the aot.map file based on the current aot cache file.
>
> This was a side effect of something else I was testing, but I can try to
> find a simpler test case if you can't reproduce it in your own testing apps.
>
> On Thu, Oct 9, 2025 at 5:24 PM Vladimir Kozlov <vladimir.kozlov at oracle.com>
> wrote:
>
>> On 10/9/25 8:16 AM, Vladimir Kozlov wrote:
>> > Hi María,
>> >
>> > Please provide command lines you use in your experiments.
>> >
>> > > And then I did a production run using the generated cache,
>> which
>> > > replaced the `aot.map` with a new one that is much smaller:
>> >
>> > We don't support incremental update of AOT cache file. What do you mean
>> > by "replaced the `aot.map`" ?
>>
>> Sorry, I confused aot.map with aot.cache.
>>
>> I assume aot.map is log file produced from AOT cache logs output.
>>
>> Still it would be interesting to see your commands.
>>
>> Vladimir K
>>
>> >
>> > Thanks,
>> > Vladimir K
>> >
>> > On 10/9/25 12:25 AM, María Arias de Reyna Dominguez wrote:
>> >> Hi Ashutosh,
>> >>
>> >> Training run:
>> >> $ grep '@@ Class ' aot.map | wc -l
>> >> 10884
>> >>
>> >> Production run:
>> >> $ grep '@@ Class ' aot.map | wc -l
>> >> 3431
>> >>
>> >> And the sizes of the map file are different:
>> >> $ du -sh aot.map*
>> >> 331M aot.map
>> >> 391M aot.map.0
>> >> 394M aot.map.1
>> >>
>> >> The aot.map is the production run. The file aot.map.1 is the training
>> >> run.
>> >>
>> >> The aot.map.0 is for app.aot.config (app.aot is how I call my aot
>> cache):
>> >> $ cat aot.map.0 | head
>> >> AOT cache map for app.aot.config
>> >> [header 0x0000000000000000 - 0x00000000000003d8 984
>> >> bytes]
>> >> $ grep '@@ Class ' aot.map.0 | wc -l
>> >> 10606
>> >>
>> >> I'm using a locally build version of Java, that's true:
>> >> $ java --version
>> >> openjdk 26-internal 2026-03-17
>> >> OpenJDK Runtime Environment (build 26-internal-adhoc.delawen.jdk)
>> >> OpenJDK 64-Bit Server VM (build 26-internal-adhoc.delawen.jdk, mixed
>> >> mode, sharing)
>> >>
>> >> Build from:
>> >> $ git log | head
>> >> commit 4b4d0cd35a32448e4b056109c502af2765766432
>> >> Author: Johny Jose <johny.jose at oracle.com <mailto:
>> johny.jose at oracle.com>>
>> >> Date: Tue Oct 7 13:13:42 2025 +0000
>> >>
>> >> 8365398: TEST_BUG: java/rmi/transport/checkLeaseInfoLeak/
>> >> CheckLeaseLeak.java failing intermittently
>> >>
>> >> Reviewed-by: msheppar, smarks, jpai
>> >>
>> >> I'm confused :)
>> >>
>> >> On Wed, Oct 8, 2025 at 6:31 PM Ashutosh Mehra <asmehra at redhat.com
>> >> <mailto:asmehra at redhat.com>> wrote:
>> >>
>> >> Hi Maria,
>> >> This is strange. I tried the same using the JavacBench app and
>> >> didn't see this discrepancy in the map file generated by the
>> >> assembly phase and the production run.
>> >> You may have already checked it but just to confirm if run "grep
>> '@@
>> >> Class ' t.log | wc -l" on the map file, do you see the same
>> >> difference?
>> >>
>> >> Thanks,
>> >> - Ashutosh Mehra
>> >>
>> >>
>> >> On Wed, Oct 8, 2025 at 4:36 AM María Arias de Reyna Dominguez
>> >> <mariasde at redhat.com <mailto:mariasde at redhat.com>> wrote:
>> >>
>> >> Hi!
>> >>
>> >> I have a question. I did a training run and generated what
>> >> looks like a valid cache.
>> >>
>> >> Summary of the number of elements (for certain types of
>> >> elements) the map contains:
>> >> ```
>> >> Classes in AOT Cache: 10.885
>> >> Methods in AOT Cache: 121.393
>> >> -> ConstMethods: 121.293
>> >> -> MethodCounters: 6.530
>> >> -> MethodData: 4.127
>> >> ConstantPool: 10.197
>> >> -> ConstantPoolCache: 10.197
>> >> ```
>> >>
>> >> And then I did a production run using the generated cache,
>> which
>> >> replaced the `aot.map` with a new one that is much smaller:
>> >>
>> >> ```
>> >> Classes in AOT Cache: 3.431
>> >> Methods in AOT Cache: 38.359
>> >> -> ConstMethods: 38.359
>> >> -> MethodCounters: 0
>> >> -> MethodData: 0
>> >> ConstantPool: 3.067
>> >> -> ConstantPoolCache: 3.067
>> >> ```
>> >>
>> >> As you can see, there are a lot of missing elements!
>> >>
>> >> If I check the md5sum of the actual cache file before and after
>> >> the production run, it stays the same. It is the AOT Cache Map
>> >> which changes. Is that... supposed to happen?
>> >>
>> >> Maybe it is a bug related to https://bugs.openjdk.org/browse/
>> >> JDK-8362566 <https://bugs.openjdk.org/browse/JDK-8362566> ?
>> >>
>> >> Both map files start with `AOT cache map for app.aot`, it is
>> not
>> >> the map file for the `app.aot.config` that appears during the
>> >> training run.
>> >>
>> >> Both the training and the run were the same: just spin up an
>> >> Infinispan server and shut it down after a few seconds. But I
>> >> guess that's irrelevant because the aot cache file is exactly
>> >> the same.
>> >>
>> >> I could stop generating the aot map file on the production run
>> >> because there is no need for it, but I forgot to remove it...
>> >> and then this happened.
>> >>
>> >> Kind regards,
>> >> María Arias de Reyna Domínguez
>> >> Senior Software Engineer
>> >> She / Her / Hers
>> >> ariasdereyna at redhat.com <mailto:ariasdereyna at redhat.com>
>> >>
>> >
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/leyden-dev/attachments/20251009/d42e1d82/attachment.htm>
More information about the leyden-dev
mailing list