August 2024 - leyden-dev

Questions about the Hermetic Java project
by Magnus Ihse Bursie 09 Jun '25

09 Jun '25

First some background for build-dev: I have spent some time looking at the build implications of the Hermetic Java effort, which is part of Project Leyden. A high-level overview is available here: https://cr.openjdk.org/~jiangli/hermetic_java.pdf and the current source code is here: https://github.com/openjdk/leyden/tree/hermetic-java-runtime. Hermetic Java faces several challenges, but the part that is relevant for the build system is the ability to create static libraries. We've had this functionality (in three different ways...) for some time, but it is rather badly implemented. As a result of my investigations, I have a bunch of questions. :-) I have gotten some answers in private discussion, but for the sake of transparency I will repeat them here, to foster an open dialogue. 1. Am I correct in understanding that the ultimate goal of this exercise is to be able to have jmods which include static libraries (*.a) of the native code which the module uses, and that the user can then run a special jlink command to have this linked into a single executable binary (which also bundles the *.class files and any additional resources needed)? 2. If so, is the idea to create special kinds of static jmods, like java.base-static.jmod, that contains *.a files instead of lib*.so files? Or is the idea that the normal jmod should contain both? 3. Linking .o and .a files into an executable is a formidable task. Is the intention to have jlink call a system-provided ld, or to bundle ld with jlink, or to reimplement this functionality in Java? 4. Is the intention is to allow users to create their own jmods with static libraries, and have these linked in as well? This seems to be the case. If that is so, then there will always be the risk for name collisions, and we can only minimize the risk by making sure any global names are as unique as possible. 5. The original implementation of static builds in the JDK, created for the Mobile project, used a configure flag, --enable-static-builds, to change the entire behavior of the build system to only produce *.a files instead of lib*.so. In contrast, the current system is using a special target instead. In my eyes, this is a much worse solution. Apart from the conceptual principle (if the build should generate static or dynamic libraries is definitely a property of what a "configuration" means), this makes it much harder to implement efficiently, since we cannot make changes in NativeCompilation.gmk, where they are needed. That was not as much a question as a statement. 🙂 But here is the question: Do you think it would be reasonable to restore the old behavior but with the new methods, so that we don't use special targets, but instead tells configure to generate static libraries? I'm thinking we should have a flag like "--with-library-type=" that can have values "dynamic" (which is default), "static" or "both". I am not sure if "both" are needed, but if we want to bundle both lib*.so and *.a files into a single jmod file (see question 2 above), then it definitely is. In general, the cost of producing two kinds of libraries are quite small, compared to the cost of compiling the source code to object files. Finally, I have looked at how to manipulate symbol visibility. There seems many ways forward, so I feel confident that we can find a good solution. One way forward is to use objcopy to manipulate symbol status (global/local). There is an option --localize-symbol in objcopy, that has been available in objcopy since at least 2.15, which was released 2004, so it should be safe to use. But ideally we should avoid using objcopy and do this as part of the linking process. This should be possible to do, given that we make changes in NativeCompilation.gmk -- see question 5 above. As a fallback, it is also possible to rename symbols, either piecewise or wholesale, using objcopy. There are many ways to do this, using --prefix-symbols, --redefine-sym or --redefine-syms (note the -s, this takes a file with a list of symbols). Thus we can always introduce a "post factum namespace" by renaming symbols. So in the end, I think it will be fully possible to produce .a files that only has global symbols for the functions that are part of the API exposed by that library, and have all other symbols local, and make this is in a way that is consistent with the rest of the build system. Finally, a note on Hotspot. Due to debugging reasons, we export basically all symbols in hotspot as global. This is not reasonable to do for a static build. The effect of not exporting those symbols will be that SA will not function to 100%. On the other hand, I have no idea if SA works at all with a static build. Have you tested this? Is this part of the plan to support, or will it be officially dropped for Hermetic Java? /Magnus

7 38

AOTHolder pattern for caching heap objects
by ioi.lam＠oracle.com 17 Sep '24

17 Sep '24

For the development of JDK-8293336 [1], I am experimenting with a pattern of Java code to indicate what heap objects should be stored in the AOT cache. This code has been pushed to the premain branch. It looks like this [2]: class java.lang.invoke.MethodType { static class AOTHolder { private static final @Stable MethodType[] objectOnlyTypes = new MethodType[20]; private static @Stable HashMap<MethodType,MethodType> archivedMethodTypes; } Essentially the class MethodType$AOTHolder will be stored in its "already initialized" state in the AOT cache. In the production run, MethodType$AOTHolder.<clinit> will NOT be run. Its static fields are already initialized and point to the objects that were created during the assembly phase. Currently, the JVM is hard-wired to recognize only a few known AOTHolder classes. This pattern is used strictly for the AOT-caching of MethodType.internTable. I want to hear people's reaction to this. I think other internal JDK classes can use a similar pattern as well. (In the long term, we will need a more generalized way (a.k.a. the "Scrambled Egg API"), but I need to have a mechanism that works for the short term). *** The reason I needed to use the AOTHolder pattern is to remove this horrible hack [3]. (It's now removed from the premain branch). You can see more info in the comments of [3], but the gist is: - The old way of CDS heap object caching depends on re-executing the <clinit> methods of classes that want to cache heap objects - These classes would call CDS.initializeFromArchive() to fetch the cached objects. - Before CDS.initializeFromArchive() returns the objects, it wants to initialize all the classes of these objects (or else the caller would end up with an object whose class is not yet initialized) - This scheme breaks down when several classes want to cache heap objects, and their <clinit> intertwine with each other. The old code in [3] tries to work around a NullPointerException that happens due to the intertwining. That is certainly not a sustainable approach. *** The AOTHolder pattern tries to accomplish two things - Clearly identify what objects should be AOT-cached. In the case of MethodType.java, I moved only two of its many fields into its AOTHolder inner class. - Cache the static fields in all the AOTHolder classes as a single snapshot. Avoid the need to run <clinit> so we don't need to worry about execution order, etc. ====== [1] https://bugs.openjdk.org/browse/JDK-8293336 [2] https://github.com/openjdk/leyden/blob/e33ecb3ec42d142569227b4bd1987124e555… [3] https://github.com/openjdk/leyden/blob/3a84df9d9860e743684e335282e3910b14cc…

2 2

Windows build of Leyden EA
by ski n 04 Sep '24

04 Sep '24

I just watched the presentation at JVMLS on the status of project Leyden on YouTube: https://www.youtube.com/watch?v=OOPSU4LnKg0 In the talk, they pointed at the EA that came out last June. Currently, only Linux and Mac builds are available. When will the next EA come available, and will this also contain a Windows build? Raymond

2 2

git: openjdk/leyden: premain: Experimental feature to allow special instrumentation code to be executed during training run. See CacheOnlyClassesIn.java for details
by duke 30 Aug '24

30 Aug '24

Changeset: 9bab6122 Branch: premain Author: iklam <ilam3388(a)gmail.com> Date: 2024-08-29 22:10:30 +0000 URL: https://git.openjdk.org/leyden/commit/9bab6122728d62095207026a07ca5c49fac9d… Experimental feature to allow special instrumentation code to be executed during training run. See CacheOnlyClassesIn.java for details ! src/hotspot/share/cds/cds_globals.hpp ! src/hotspot/share/classfile/classLoader.cpp ! src/hotspot/share/classfile/classLoaderExt.cpp ! src/hotspot/share/classfile/classLoaderExt.hpp ! src/hotspot/share/classfile/systemDictionaryShared.cpp ! src/hotspot/share/utilities/classpathStream.hpp + test/hotspot/jtreg/runtime/cds/appcds/leyden/CacheOnlyClassesIn.java

1 0

git: openjdk/leyden: hermetic-java-runtime: 33 new changesets
by duke 29 Aug '24

29 Aug '24

Changeset: 8e88da05 Branch: hermetic-java-runtime Author: Tejesh R <tr(a)openjdk.org> Date: 2024-08-28 04:43:10 +0000 URL: https://git.openjdk.org/leyden/commit/8e88da05b9966892e117b779d59a2e311a557… 8338041: Keyboard Navigation of JTable, Ctrl Shift RIGHT/LEFT doesn't follow native action in GTK L&F Reviewed-by: honkar, prr, abhiscxk ! src/java.desktop/share/classes/com/sun/java/swing/plaf/gtk/GTKLookAndFeel.java + test/jdk/javax/swing/plaf/gtk/JTableCtrlShiftRightLeftKeyTest.java Changeset: 2e174c63 Branch: hermetic-java-runtime Author: Jaikiran Pai <jpai(a)openjdk.org> Date: 2024-08-28 09:29:18 +0000 URL: https://git.openjdk.org/leyden/commit/2e174c6367c7755d8541f9669f7f10ed89878… 8338445: jdk.internal.loader.URLClassPath may leak JarFile instance when dealing with unexpected Class-Path entry in manifest Reviewed-by: michaelm, cstein, alanb ! src/java.base/share/classes/jdk/internal/loader/URLClassPath.java + test/jdk/java/net/URLClassLoader/JarLoaderCloseTest.java Changeset: 1ff9ac72 Branch: hermetic-java-runtime Author: Maurizio Cimadamore <mcimadamore(a)openjdk.org> Date: 2024-08-28 10:22:34 +0000 URL: https://git.openjdk.org/leyden/commit/1ff9ac7233d51a58fd54a92d2c45761478574… 8338731: MemoryLayout::offsetHandle can return a negative offset Reviewed-by: pminborg, psandoz ! src/java.base/share/classes/java/lang/foreign/MemoryLayout.java ! src/java.base/share/classes/jdk/internal/foreign/LayoutPath.java ! test/jdk/java/foreign/TestLayoutPaths.java Changeset: 21505216 Branch: hermetic-java-runtime Author: Nizar Benalla <nbenalla(a)openjdk.org> Committer: Pavel Rappo <prappo(a)openjdk.org> Date: 2024-08-28 11:01:15 +0000 URL: https://git.openjdk.org/leyden/commit/2150521650d6b730cfe9d3ecb91d589c96862… 8322036: Improve help output from the javadoc tool Reviewed-by: prappo ! src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/resources/standard.properties ! src/jdk.javadoc/share/classes/jdk/javadoc/internal/tool/ToolOptions.java ! src/jdk.javadoc/share/classes/jdk/javadoc/internal/tool/resources/javadoc.properties Changeset: 9d183bd0 Branch: hermetic-java-runtime Author: Markus Grönlund <mgronlun(a)openjdk.org> Date: 2024-08-28 12:19:58 +0000 URL: https://git.openjdk.org/leyden/commit/9d183bd02763ee4ff5aa8388e039d8b5a6964… 8339149: jfr_flush_event_writer - return value type mismatch Reviewed-by: egahlin ! src/hotspot/share/jfr/jni/jfrJniMethod.hpp Changeset: 32c97509 Branch: hermetic-java-runtime Author: Albert Mingkun Yang <ayang(a)openjdk.org> Date: 2024-08-28 13:28:01 +0000 URL: https://git.openjdk.org/leyden/commit/32c975098521e830ce706b67e7232a007c084… 8339160: [BACKOUT] JDK-8338440 Parallel: Improve fragmentation mitigation in Full GC Reviewed-by: tschatzl ! src/hotspot/share/gc/parallel/psParallelCompact.cpp ! src/hotspot/share/gc/parallel/psParallelCompact.hpp Changeset: b6700095 Branch: hermetic-java-runtime Author: Eirik Bjørsnøs <eirbjo(a)openjdk.org> Date: 2024-08-28 15:23:50 +0000 URL: https://git.openjdk.org/leyden/commit/b6700095c018a67a55b746cd4eee763c68f53… 8338729: Retire the test jdk/java/util/zip/TestZipError.java Reviewed-by: lancea - test/jdk/java/util/zip/TestZipError.java Changeset: 379f3db0 Branch: hermetic-java-runtime Author: Daniel D. Daugherty <dcubed(a)openjdk.org> Date: 2024-08-28 16:47:30 +0000 URL: https://git.openjdk.org/leyden/commit/379f3db001fe4bffd3a00e0363a98275e7b2e… 8339175: ProblemList runtime/interpreter/LastJsrTest.java on all platforms with Xcomp Reviewed-by: matsaave ! test/hotspot/jtreg/ProblemList-Xcomp.txt ! test/hotspot/jtreg/ProblemList.txt Changeset: 0c2b1758 Branch: hermetic-java-runtime Author: Anthony Scarpino <ascarpino(a)openjdk.org> Date: 2024-08-28 17:24:33 +0000 URL: https://git.openjdk.org/leyden/commit/0c2b175898d13b58ffe56e2f9cbc9d88173a6… 8328608: Multiple NewSessionTicket support for TLS Reviewed-by: djelinski ! src/java.base/share/classes/sun/security/ssl/Finished.java ! src/java.base/share/classes/sun/security/ssl/NewSessionTicket.java ! src/java.base/share/classes/sun/security/ssl/PreSharedKeyExtension.java ! src/java.base/share/classes/sun/security/ssl/SSLConfiguration.java ! src/java.base/share/classes/sun/security/ssl/SSLEngineImpl.java ! src/java.base/share/classes/sun/security/ssl/SSLSessionContextImpl.java ! src/java.base/share/classes/sun/security/ssl/SSLSessionImpl.java ! src/java.base/share/classes/sun/security/ssl/SSLSocketImpl.java ! src/java.base/share/classes/sun/security/util/Cache.java ! test/jdk/javax/net/ssl/SSLSession/CertMsgCheck.java ! test/jdk/javax/net/ssl/SSLSession/CheckSessionContext.java ! test/jdk/javax/net/ssl/templates/TLSBase.java + test/jdk/sun/security/ssl/SSLSessionImpl/MultiNSTClient.java + test/jdk/sun/security/ssl/SSLSessionImpl/MultiNSTNoSessionCreation.java + test/jdk/sun/security/ssl/SSLSessionImpl/MultiNSTParallel.java + test/jdk/sun/security/ssl/SSLSessionImpl/MultiNSTSequence.java Changeset: 3d49fb8a Branch: hermetic-java-runtime Author: Manukumar V S <mvs(a)openjdk.org> Date: 2024-08-28 17:54:43 +0000 URL: https://git.openjdk.org/leyden/commit/3d49fb8a17ceec6e23595bc8affc89765899f… 8338103: Stabilize and open source a Swing OGL ButtonResizeTest Reviewed-by: abhiscxk, prr, tr + test/jdk/javax/swing/JButton/SwingButtonResizeTestWithOpenGL.java Changeset: a98ecad0 Branch: hermetic-java-runtime Author: Claes Redestad <redestad(a)openjdk.org> Date: 2024-08-28 18:16:00 +0000 URL: https://git.openjdk.org/leyden/commit/a98ecad0a920f12d81386de3d0f549d542014… 8338897: Small startup regression remains after JDK-8309622 and JDK-8331932 Reviewed-by: liach, naoto ! src/java.base/share/classes/java/util/Locale.java ! src/java.base/share/classes/sun/util/locale/BaseLocale.java Changeset: eff6d9cd Branch: hermetic-java-runtime Author: Claes Redestad <redestad(a)openjdk.org> Date: 2024-08-28 18:22:30 +0000 URL: https://git.openjdk.org/leyden/commit/eff6d9cd23f9da8720a44ad628aa0a3e6f58f… 8339167: Remove AbstractPoolEntry.PrimitiveEntry to reduce boxing overheads Reviewed-by: liach ! src/java.base/share/classes/jdk/internal/classfile/impl/AbstractPoolEntry.java ! src/java.base/share/classes/jdk/internal/classfile/impl/SplitConstantPool.java Changeset: d03ec7aa Branch: hermetic-java-runtime Author: Leonid Mesnik <lmesnik(a)openjdk.org> Date: 2024-08-28 20:17:25 +0000 URL: https://git.openjdk.org/leyden/commit/d03ec7aad41d830b47801b7af75ee5e278128… 8339030: frame::print_value_on(outputStream* st, JavaThread *thread) doesn't need thread argument Reviewed-by: dholmes, coleenp ! src/hotspot/share/oops/instanceStackChunkKlass.cpp ! src/hotspot/share/prims/whitebox.cpp ! src/hotspot/share/runtime/continuationFreezeThaw.cpp ! src/hotspot/share/runtime/frame.cpp ! src/hotspot/share/runtime/frame.hpp ! src/hotspot/share/runtime/javaThread.cpp ! src/hotspot/share/runtime/vframe.cpp Changeset: d08b5bd9 Branch: hermetic-java-runtime Author: Leonid Mesnik <lmesnik(a)openjdk.org> Date: 2024-08-28 20:18:51 +0000 URL: https://git.openjdk.org/leyden/commit/d08b5bd9f5f740d75c1acfbd644ce1c822e03… 8258483: [TESTBUG] gtest CollectorPolicy.young_scaled_initial_ergo_vm fails if heap is too small Reviewed-by: ayang ! test/hotspot/gtest/gc/shared/test_collectorPolicy.cpp Changeset: a8ac2872 Branch: hermetic-java-runtime Author: Justin Lu <jlu(a)openjdk.org> Date: 2024-08-28 21:14:29 +0000 URL: https://git.openjdk.org/leyden/commit/a8ac28725bfc22867c76856ddce094588a97b… 8339126: JNI exception pending in Inflater.c Reviewed-by: lancea, vtewari, jpai, naoto ! src/java.base/share/native/libzip/Inflater.c Changeset: 72a49005 Branch: hermetic-java-runtime Author: David Holmes <dholmes(a)openjdk.org> Date: 2024-08-28 21:16:18 +0000 URL: https://git.openjdk.org/leyden/commit/72a49005ee8c4aeb6dcf3eff4c56576a2b4d0… 8338888: SystemDictionary::class_name_symbol has incorrect length check Reviewed-by: stuefe, kbarrett, coleenp ! src/hotspot/share/classfile/systemDictionary.cpp ! test/hotspot/jtreg/runtime/exceptionMsgs/NoClassDefFoundError/NoClassDefFoundErrorTest.java ! test/hotspot/jtreg/runtime/exceptionMsgs/NoClassDefFoundError/libNoClassDefFoundErrorTest.c Changeset: 26e3d535 Branch: hermetic-java-runtime Author: Brent Christian <bchristi(a)openjdk.org> Date: 2024-08-28 22:54:38 +0000 URL: https://git.openjdk.org/leyden/commit/26e3d535ad4d6e5d78ca50941cfa39dd33789… 8338716: Re-visit "interrupt handling" in jdk.internal.loader.Resource Reviewed-by: alanb ! src/java.base/share/classes/jdk/internal/loader/Resource.java Changeset: 0ddcd701 Branch: hermetic-java-runtime Author: Dean Long <dlong(a)openjdk.org> Date: 2024-08-29 00:34:11 +0000 URL: https://git.openjdk.org/leyden/commit/0ddcd7017576a0f9c97a74b7d47624ae06ed0… 8335120: assert(!target->can_be_statically_bound() || target == cha_monomorphic_target) failed Reviewed-by: kvn, vlivanov ! src/hotspot/share/c1/c1_GraphBuilder.cpp ! src/hotspot/share/ci/ciMethod.cpp ! src/hotspot/share/ci/ciMethod.hpp Changeset: eb7ead58 Branch: hermetic-java-runtime Author: Prasanta Sadhukhan <psadhukhan(a)openjdk.org> Date: 2024-08-29 05:03:15 +0000 URL: https://git.openjdk.org/leyden/commit/eb7ead58fd70822669d2aa1a0053814e58955… 8336873: BasicSplitPaneDivider:oneTouchExpandableChanged() should mention that implementation depends on SplitPane.supportsOneTouchButtons property Reviewed-by: prr, abhiscxk ! src/java.desktop/share/classes/javax/swing/plaf/basic/BasicSplitPaneDivider.java Changeset: 1383fec4 Branch: hermetic-java-runtime Author: Kangcheng Xu <kxu(a)openjdk.org> Date: 2024-08-29 05:34:08 +0000 URL: https://git.openjdk.org/leyden/commit/1383fec41756322bf2832c55633e46395b937… 8327381: Refactor type-improving transformations in BoolNode::Ideal to BoolNode::Value Reviewed-by: chagedorn, thartmann, jkarthikeyan, epeter ! src/hotspot/share/opto/subnode.cpp ! src/hotspot/share/opto/subnode.hpp + test/hotspot/jtreg/compiler/c2/gvn/TestBoolNodeGVN.java ! test/hotspot/jtreg/compiler/lib/ir_framework/IRNode.java Changeset: 0b4a7d53 Branch: hermetic-java-runtime Author: Jan Lahoda <jlahoda(a)openjdk.org> Date: 2024-08-29 06:25:27 +0000 URL: https://git.openjdk.org/leyden/commit/0b4a7d534204b7b3b041f5117282dd13b1c7c… 8324859: Improve error recovery Reviewed-by: mcimadamore ! src/jdk.compiler/share/classes/com/sun/tools/javac/parser/JavacParser.java ! test/langtools/tools/javac/parser/JavacParserTest.java Changeset: ff59532d Branch: hermetic-java-runtime Author: Jan Lahoda <jlahoda(a)openjdk.org> Date: 2024-08-29 06:28:05 +0000 URL: https://git.openjdk.org/leyden/commit/ff59532ddd3002df61e46d58b3f29d26c7829… 8338678: Erroneous parameterized type represented as <any> Reviewed-by: vromero ! src/jdk.compiler/share/classes/com/sun/tools/javac/code/Type.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/Attr.java ! test/langtools/tools/javac/recovery/AttrRecovery.java Changeset: f080b4bb Branch: hermetic-java-runtime Author: Matthias Baesken <mbaesken(a)openjdk.org> Date: 2024-08-29 07:07:12 +0000 URL: https://git.openjdk.org/leyden/commit/f080b4bb8a75284db1b6037f8c00ef3b1ef1a… 8333098: ubsan: bytecodeInfo.cpp:318:59: runtime error: division by zero Reviewed-by: kvn, iveresov ! src/hotspot/share/opto/bytecodeInfo.cpp Changeset: 362f9ce0 Branch: hermetic-java-runtime Author: Magnus Ihse Bursie <ihse(a)openjdk.org> Date: 2024-08-29 07:29:12 +0000 URL: https://git.openjdk.org/leyden/commit/362f9ce077baa900ed81a0473ec0187efde13… 8339120: Use more fine-granular gcc unused warnings Reviewed-by: jwaters, kbarrett, erikj ! make/autoconf/flags-cflags.m4 ! make/common/TestFilesCompilation.gmk ! make/common/modules/LauncherCommon.gmk ! make/hotspot/lib/CompileGtest.gmk ! make/hotspot/lib/CompileJvm.gmk ! make/modules/java.base/Lib.gmk ! make/modules/java.base/lib/CoreLibraries.gmk ! make/modules/java.desktop/Lib.gmk ! make/modules/java.desktop/lib/AwtLibraries.gmk ! make/modules/java.desktop/lib/ClientLibraries.gmk ! make/modules/java.management/Lib.gmk ! make/modules/java.security.jgss/Lib.gmk ! make/modules/jdk.crypto.cryptoki/Lib.gmk ! make/modules/jdk.hotspot.agent/Lib.gmk ! make/modules/jdk.jdwp.agent/Lib.gmk ! make/modules/jdk.jpackage/Lib.gmk ! make/modules/jdk.management/Lib.gmk Changeset: 723588a4 Branch: hermetic-java-runtime Author: Daniel Fuchs <dfuchs(a)openjdk.org> Date: 2024-08-29 08:54:02 +0000 URL: https://git.openjdk.org/leyden/commit/723588a4e78d25f0ef3c4cdaeb377aedc3a35… 8338569: HTTP/1.1 CleanupTrigger may be triggerred after the next exchange started Reviewed-by: jpai ! src/java.net.http/share/classes/jdk/internal/net/http/ConnectionPool.java ! src/java.net.http/share/classes/jdk/internal/net/http/SocketTube.java ! src/java.net.http/share/classes/jdk/internal/net/http/common/FlowTube.java ! test/jdk/java/net/httpclient/DigestEchoClient.java ! test/jdk/java/net/httpclient/ShutdownNow.java ! test/jdk/java/net/httpclient/SmokeTest.java Changeset: d35ffa4f Branch: hermetic-java-runtime Author: Andrey Turbanov <aturbanov(a)openjdk.org> Date: 2024-08-29 09:57:52 +0000 URL: https://git.openjdk.org/leyden/commit/d35ffa4f6afb7df052103cee8544e4e707b72… 8339017: Make a couple of fields in DoubleByte static Reviewed-by: bpb, naoto ! src/java.base/share/classes/sun/nio/cs/DoubleByte.java Changeset: 8c8b5801 Branch: hermetic-java-runtime Author: Jan Lahoda <jlahoda(a)openjdk.org> Date: 2024-08-29 10:06:08 +0000 URL: https://git.openjdk.org/leyden/commit/8c8b5801fd9d28a71edf3bd8d1fae857817e2… 8338281: jshell does not run shutdown hooks Reviewed-by: asotona ! src/jdk.jshell/share/classes/jdk/jshell/execution/ExecutionControlForwarder.java ! src/jdk.jshell/share/classes/jdk/jshell/execution/JdiDefaultExecutionControl.java ! test/langtools/jdk/jshell/ShutdownTest.java Changeset: e57b5932 Branch: hermetic-java-runtime Author: Johan Sjölen <jsjolen(a)openjdk.org> Date: 2024-08-29 11:23:04 +0000 URL: https://git.openjdk.org/leyden/commit/e57b59325831247818cb4b07c4fd43e4556ef… 8335062: NMT: Make StackIndex non-opaque Reviewed-by: stuefe, gziemski ! src/hotspot/share/nmt/nmtNativeCallStackStorage.hpp ! src/hotspot/share/nmt/vmatree.hpp ! test/hotspot/gtest/nmt/test_nmt_nativecallstackstorage.cpp Changeset: 777ed2b5 Branch: hermetic-java-runtime Author: Chen Liang <liach(a)openjdk.org> Date: 2024-08-29 15:45:52 +0000 URL: https://git.openjdk.org/leyden/commit/777ed2b5d2ef8371407cc9bf0370a7cef937c… 8339132: Make DirectCodeBuilder write through without allocating instruction objects Reviewed-by: asotona, redestad ! src/java.base/share/classes/java/lang/classfile/CodeBuilder.java ! src/java.base/share/classes/jdk/internal/classfile/impl/AbstractInstruction.java ! src/java.base/share/classes/jdk/internal/classfile/impl/BytecodeHelpers.java ! src/java.base/share/classes/jdk/internal/classfile/impl/DirectCodeBuilder.java Changeset: a4962ace Branch: hermetic-java-runtime Author: David Holmes <dholmes(a)openjdk.org> Date: 2024-08-29 20:38:52 +0000 URL: https://git.openjdk.org/leyden/commit/a4962ace4d3afb36e9d6822a4f02a1515fac4… 8338257: UTF8 lengths should be size_t not int Reviewed-by: stuefe, coleenp, dlong ! src/hotspot/share/classfile/compactHashtable.cpp ! src/hotspot/share/classfile/compactHashtable.hpp ! src/hotspot/share/classfile/javaClasses.cpp ! src/hotspot/share/classfile/javaClasses.hpp ! src/hotspot/share/classfile/modules.cpp ! src/hotspot/share/classfile/stringTable.cpp ! src/hotspot/share/classfile/symbolTable.cpp ! src/hotspot/share/jfr/dcmd/jfrDcmds.cpp ! src/hotspot/share/jfr/jni/jfrJavaSupport.cpp ! src/hotspot/share/jfr/recorder/checkpoint/types/jfrThreadState.cpp ! src/hotspot/share/oops/symbol.cpp ! src/hotspot/share/prims/jni.cpp ! src/hotspot/share/prims/jvmtiEnv.cpp ! src/hotspot/share/services/finalizerService.cpp ! src/hotspot/share/utilities/utf8.cpp ! src/hotspot/share/utilities/utf8.hpp Changeset: f2968b34 Branch: hermetic-java-runtime Author: Matias Saavedra Silva <matsaave(a)openjdk.org> Date: 2024-08-29 21:06:05 +0000 URL: https://git.openjdk.org/leyden/commit/f2968b34a55009fb195e381ffa615488974e9… 8339020: Remove unused HeapShared::calculate_oopmap Reviewed-by: coleenp ! src/hotspot/share/cds/heapShared.cpp ! src/hotspot/share/cds/heapShared.hpp Changeset: b711c41d Branch: hermetic-java-runtime Author: Shaojin Wen <swen(a)openjdk.org> Committer: Chen Liang <liach(a)openjdk.org> Date: 2024-08-29 21:21:16 +0000 URL: https://git.openjdk.org/leyden/commit/b711c41d442fc369a84745c0203db638e0b7e… 8339196: Optimize BufWriterImpl#writeU1/U2/Int/Long Reviewed-by: liach, redestad ! src/java.base/share/classes/jdk/internal/classfile/impl/BufWriterImpl.java Changeset: 2ea61c7b Branch: hermetic-java-runtime Author: Jiangli Zhou <jianglizhou(a)google.com> Date: 2024-08-29 16:25:53 +0000 URL: https://git.openjdk.org/leyden/commit/2ea61c7b93004b21e1950a768839e8f1e0aee… Merge branch 'master' into hermetic-java-runtime ! make/modules/java.base/lib/CoreLibraries.gmk ! make/modules/java.base/lib/CoreLibraries.gmk

1 0

git: openjdk/leyden: premain: Simplified dumping of MethodType with the AOTHolder pattern
by duke 29 Aug '24

29 Aug '24

Changeset: e33ecb3e Branch: premain Author: iklam <ilam3388(a)gmail.com> Date: 2024-08-28 18:04:19 +0000 URL: https://git.openjdk.org/leyden/commit/e33ecb3ec42d142569227b4bd1987124e5551… Simplified dumping of MethodType with the AOTHolder pattern ! src/hotspot/share/cds/aotClassInitializer.cpp ! src/hotspot/share/cds/heapShared.cpp ! src/java.base/share/classes/java/lang/invoke/DirectMethodHandle.java ! src/java.base/share/classes/java/lang/invoke/InvokerBytecodeGenerator.java ! src/java.base/share/classes/java/lang/invoke/LambdaForm.java ! src/java.base/share/classes/java/lang/invoke/MethodType.java ! test/hotspot/jtreg/TEST.groups

1 0

RFR: 8338017: Add AOT command-line flag aliases (Leyden)
by ioi.lam＠oracle.com 29 Aug '24

29 Aug '24

Hi Leyden developers, https://github.com/openjdk/jdk/pull/20516 Please review the first PR for JEP 483: Ahead-of-Time Class Loading & Linking [1]. JEP 483 is still in the Candidate stage, but I would like to get the reviews rolling so we can integrate soon after it's targeted. The plan [2] is to break down JEP 483 in a series of 7 PRs to make it easier to review. I will post the other PRs soon. Thanks - Ioi === [1] https://openjdk.org/jeps/483 [2] https://bugs.openjdk.org/browse/JDK-8331497

1 0

for the record: heat based sorting, driven by training runs
by John Rose 28 Aug '24

28 Aug '24

The following thoughts are “for the record” not “for immediate implementation”. They are an example of good ideas that come up in our conversations, are given some thought, and are then prioritized down the list. For later. Maybe even for never. The important thing is to have a reasonably complete record, so we don’t have to rediscover and rediscuss such points. There are ever so very many of such points, we would be at it forever rediscovering and rediscussing. With that said… It is well known that some workloads (in many languages) sometimes benefit from changing the order of fields in structs (or objects or whatever they are called in whatever language). In simple terms, if X.A is accessed very often, and X.Z is accessed seldom, then there is little or no penalty to putting them on opposite ends of a struct layout, even though the struct might lie across a data cache boundary, so that X.A and X.Z must incur distinct data cache fill events. The same point is true if the struct is used heavily in two different ways (different program phases), so that X.Z is sometimes accessed frequently, but never near the time when X.A was accessed frequently. A similar point holds if X.A and X.B are always accessed together. You don’t want them to be far away from each other, far enough to make it likely that they end up on distinct cache lines (requiring double the number of fill events for a use of both). For Java, the object header is, for some classes, accessed very frequently indeed. And for others it is almost never accessed, except during creation and GC! It depends on whether the class in question is used solely as a static type. There are plenty of examples; the internal nodes of hash tables work that way. In any case, the header works like just another field X.H, in this analysis, and may be accessed seldom or frequently, and coupled with other fields, or independently. There is a distinct NP-hard feel about the optimization problems here, and yet very simple tactics can lead to solid benefits (often in the single digit percentage range). Sorting fields by frequency of access is well known. It covers the case where X.A and X.B are heavily used together. It doesn’t cover the case where both are heavily used but at different times. That doesn’t matter; it covers enough cases to be beneficial as a standard move. A similar standard move is sorting functions in a code heap by hotness. You don’t necessarily have to look at their caller/callee interrelationships, although that’s a next step. And a third similar standard move is to sort the basic blocks within a function, again by hotness. For that one, you need to respect their intrinsic in/out relations, from the start. What I say about struct layout can be cross-applied to these other problem areas as well. Let’s just call all of them “heat based sorting”, whether or not the heat is the primary sort key or whether it is a “secondary” influence on the resulting order. A good practical study of how heat-based sorting can help is here: https://dl.acm.org/doi/abs/10.1109/CGO51591.2021.9370314 This is the Facebook HipHop VM, not a JVM. But some of their experiences are relevant to us. For object layout, the need for holes to get word alignment will make a pure heat-based sort impractical, but a simple heat-based sorting pass, followed by some hole-filling that violates the heat-based order, might work out fine. The object header should probably go next to the hottest field, in most cases. But in cases where the object header is cold, the object layout could start with cold fields. In order to make this work, the training run would have to gather heat statistics. I imagine a per-class array of counters, indexed by offset (byte offset) within the class’s current object layout. Reading or writing a field (or the header) increments the counter. (Extra points for probabilistic or indirect counters which don’t drive the memory fabric insane, making the application dynamics non-representative of real, non-instrumented deployments. Heisenberg and all that.) The assembly phase will then take stock of the training run outputs and re-load everything into a nicely curated bundle for the production runs. As it reloads classes, it evaluates whether their layouts should change (because of interesting observed dynamics). Not all should change, but some might, if they have enough variance across their field statistics. At that point it is a “mere matter of programming” to set up the adjusted (heat-based) layouts. The Valhalla project is adding extra “programmability” to layout logic, so perhaps we will have some new tools from them. (Thanks, Fredric.) A similar point goes for function ordering in the code cache, on that day (when it comes) when the assembly phase gets fine-grained control of function placement in the code cache. Just as alignment considerations require us to modify a pure heat-based sorting principle, it may be better to put related functions (of approximately similar heat) back-to-back, even if the pure heat-based sorting would shuffle together unrelated functions. A similar point goes for basic-block sorting, again with caveats about putting sequential blocks in sequence, unless there is evidence that sequence has no benefit (because of large variance in heat). In the case of basic-block sorting, we have a sticky problem (noted by the HipHop people) that regeneration of code will probably obliterate basic block identities. In order to reorder basic blocks (in the output phase, for example, which is where C2 chooses order), the ordering heuristics need to access profile data for a previous occurrence (in a previous run) of that SAME block. We don’t currently have a block “sameness” criterion. HipHop does this by having a very low level BB-based IR, from which they assemble twice, once with profile counters, and once USING those counters. We could do this (perhaps with deep C2 cuts) or we could contrive to “name” our basic blocks in such a way that they OFTEN (perhaps not always) correspond to IR generation tasks that work from a common source (classfile, inline tree) but do not necessarily create exactly the same set of basic blocks. That might be enough, depending the “naming scheme” we come up with to refer to basic blocks as they are generated (during training for profiling) and later on a second time (during assembly). I don’t have a clear solution to propose yet, but that is the way I see this particular problem space. Although it is not urgent yet, at some point somebody (maybe a graduate student?) might wish to try out a targeted experiment with field reordering, and see what pops out. We don’t have time to depend on this now, and we have enough high-priority (cheaper and more rewarding) tasks to fill our plate. But, these are my thoughts for the record… — John

1 1

git: openjdk/leyden: premain: 2 new changesets
by duke 28 Aug '24

28 Aug '24

Changeset: 1bd62091 Branch: premain Author: iklam <ilam3388(a)gmail.com> Date: 2024-08-26 21:32:09 +0000 URL: https://git.openjdk.org/leyden/commit/1bd6209119e17dcd17c294b8b3441224ad231… Simplified the archiving of enum class mirrors ! src/hotspot/share/cds/archiveBuilder.cpp ! src/hotspot/share/cds/cdsEnumKlass.cpp ! src/hotspot/share/cds/cdsEnumKlass.hpp ! src/hotspot/share/cds/classPreinitializer.cpp ! src/hotspot/share/cds/heapShared.cpp ! src/hotspot/share/cds/heapShared.hpp ! src/hotspot/share/runtime/threads.cpp ! test/hotspot/jtreg/runtime/cds/appcds/cacheObject/ArchiveHeapTestClass.java Changeset: 3a84df9d Branch: premain Author: iklam <ilam3388(a)gmail.com> Date: 2024-08-27 22:44:57 +0000 URL: https://git.openjdk.org/leyden/commit/3a84df9d9860e743684e335282e3910b14cc9… Renamed classPreinitializer.cpp to aotClassInitializer.cpp; more heapShared.cpp clean up + src/hotspot/share/cds/aotClassInitializer.cpp = src/hotspot/share/cds/aotClassInitializer.hpp ! src/hotspot/share/cds/cdsConfig.hpp ! src/hotspot/share/cds/cdsHeapVerifier.cpp - src/hotspot/share/cds/classPreinitializer.cpp ! src/hotspot/share/cds/heapShared.cpp ! src/hotspot/share/cds/heapShared.hpp ! src/hotspot/share/cds/metaspaceShared.cpp ! src/hotspot/share/classfile/systemDictionaryShared.cpp ! src/hotspot/share/oops/instanceKlass.cpp ! test/hotspot/jtreg/runtime/cds/appcds/cacheObject/ArchiveHeapTestClass.java

1 0

EA feedback
by Danny Thomas 28 Aug '24

28 Aug '24

I tried 24-leydenpremain+2-8 on a few internal applications, some quick feedback below (good to see you folks at the JVM LS!). If a jar has a Class-Path attribute and one or more of those libraries are explicitly on the classpath, it causes the actual and expected classpath to always differ. This is also the case currently with CDS of course, but this feature is sure to be deployed far more broadly than CDS is currently, so likely something you want to look at: [0.057s][info][class,path] non-existent Class-Path entry lib/failureaccess-1.0.1.jar [0.057s][info][class,path] opened: lib/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar [0.057s][info][class,path] library = lib/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar Startup time when training seems to be on par with ArchiveClassesAtExit in JDK 21, but it's about a 3.5x startup time penalty for one of our typical Spring Boot applications. From a back-to-back run on my machine (AMD EPYC 9R14, 32 cores, 123G, Ubuntu 22.04.4 LTS): Started App in 7.698 seconds (process running for 8.229) Started App in 26.247 seconds (process running for 29.262) - w/ CacheDataStore, Training Run Started App in 4.341 seconds (process running for 4.917) - w/ CacheDataStore, Production Run I also got a crash on one attempt, I can't remember what I did to cause this unfortunately: Stack: [0x00007f3949ab0000,0x00007f3949bb0000], sp=0x00007f3949bae628, free space=1017k Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code) V [libjvm.so+0x42ca30] ArchiveBuilder::get_buffered_addr(unsigned char*) const+0x40 V [libjvm.so+0xce4aa5] VM_PopulateDumpSharedSpace::doit()+0x395 V [libjvm.so+0x100ae69] VM_Operation::evaluate()+0x109 V [libjvm.so+0x100e348] VMThread::evaluate_operation(VM_Operation*)+0xe8 V [libjvm.so+0x10142fb] VMThread::inner_execute(VM_Operation*)+0x35b V [libjvm.so+0x101460f] VMThread::run()+0x16f V [libjvm.so+0xf6e5cf] Thread::call_run()+0x9f V [libjvm.so+0xd74e13] thread_native_entry(Thread*)+0x183 C [libc.so.6+0x98b07] siginfo: si_signo: 11 (SIGSEGV), si_code: 1 (SEGV_MAPERR), si_addr: 0x0000000000000030 Thinking ahead to operationalizing AOT, while a single-shot/on-exit workflow is great for iterating locally, requiring the VM to exit makes this more difficult to operationalize at scale: 1. We'll perform training and assembly on test, production canary and production instances on behalf of application owners and handle distribution of the archives. Depending on when we're able to perform a training run, it'll have different benefits. i.e.: 1. Test environment will at least improve startup performance, with a mixed benefit for warm up depending on the kind of traffic they take in test 2. If an application uses canary deployments we'll have a full production profile prior to the full production deployment, and all instances will come up hot 3. If we reach production with only a test environment profile, we'll perform a training run in production, so instances that scale up following that run will come up hot (completely cold instances for an initial deployment is less of a concern, because we deploy immutably and get a natural warm-up period while we have 200% capacity online for a cluster) 2. It's currently not a problem if a VM doesn't exit completely due to a dangling non-daemon thread or hung shutdown hook Being able to trigger assembly/verification via jcmd without exiting, would make this far easier for us to support. If the overhead of the instrumentation for CDS can be avoided, being able to take a snapshot at any time on any VM would be better still, but that wouldn't be an impediment for us: we'll know that the instance will be used for training at boot time. We build nightlies of all the currently active OpenJDK projects, so if you land anything on premain between EA builds that you'd like us to try, let us know! Cheers, Danny

8 11