November 2024 - leyden-dev

[RFR] 8344140: Refactor the discovery of AOT cache artifacts
by ioi.lam＠oracle.com 22 Nov '24

22 Nov '24

FYI - This is a PR for the JDK mainline related to Leyden development. Please comment on the PR page: https://github.com/openjdk/jdk/pull/22291 ==================== This is a clean up after the initial integration for JEP 483 + Merged 3 loops into 1 for discovering the classes and oops that should be stored in the AOT cache. About 250 lines of code are deleted. + Added comments in aotArtifactFinder.hpp to describe the logic, which is much simpler than before. + Enum classes are no longer unconditionally AOT-initialized -- an Enum class is AOT-initialized only if we find an archived oop of this type. Note to reviewers: One consequence of this refactoring is that archived oops are now discovered (by heapShared.cpp) before the metadata are copied (by ArchiveBuilder). There are some functions that depend on the old order (metadata were copied before oop discovery), so I had to shuffle them around. Examples are Modules::check_archived_module_oop(), or the new code in Klass::remove_java_mirror().

1 0

git: openjdk/leyden: created branch premain-temp-merge-after-jep-483 based on the branch master containing 495 unique commits
by duke 22 Nov '24

22 Nov '24

The following commits are unique to the premain-temp-merge-after-jep-483 branch: ======================================================== 626b62aa: Initial commit f5321dd8: Merge branch 'master' of https://github.com/openjdk/leyden into premain d47df36e: More relocInfo fixes for aarch64 49d035dc: Additional card table fixes 3daab89b: 8314078: HotSpotConstantPool.lookupField() asserts due to field changes in ConstantPool.cpp 6825ecde: another relocInfo patching fix e6b1a385: Disabled some CDS test code affected by the earleir work-around of JDK-8307468 in leyden-premain e0562723: Added information in the CDS map file for the permobj segments used by HeapShared::get_archived_object_permanent_index() 678d7f91: Added test/hotspot/jtreg/premain/javac_helloworld start-up benchmark ff3c770b: Temp work-around for some test cases that see two classes named "jdk.proxy2.$Proxy8" 78194355: Skip SC generation for custom class loaders f3073178: Merge branch 'master' of https://github.com/openjdk/leyden into premain cc41a022: 8314846: Do not store Klass::_secondary_super_cache in CDS archive 1d1854ed: fixed missing JvmtiExport::_should_notify_object_alloc address when AOT compiling javac with -XX:+ArchiveInvokeDynamic 631e2f0f: improve logging of (cds+resolve) 980b5d92: Updated field CP archiving to match JDK-8301996 4ef082a9: 8314247: JVMCI: expected int64_t but JavaThread::_held_monitor_count is of type intx (this is temp workaround from the withdrawn https://github.com/openjdk/jdk/pull/15295) e1a591e7: Check recompilation schedule existence to avoid potential crashes if the 3ed4a9e3: ConstantPool::remove_unshareable_info() clean up 3910f790: Fixed premain docs 38387c5b: Fix MethodTrainingData::make 9b92dfc2: removed unused SystemDictionaryShared::init_dumptime_info(Method* m) d1b61107: 8315431: ArchiveHeapWriter::get_filler_size_at() cannot handle buffer expansion fa813084: Renames in cached code. fd82682f: Rename class SCArchive to SCCache d960fb15: Work around for JDK-8315719: [premain] -XX:+ArchiveInvokeDynamic is incompatible with old classes 265c226c: Fixed CDS jtreg tests to use new command line options like -XX:+StoreCachedCode 7b4a856f: cds: archive unique exception instances c95d6058: New "one step training run" workflow with -XX:CacheDataStore d1048945: Leyden-premain + spring-petclinic demo c5b1038e: fixed typos in tests a8280594: Fix processing cache file path f4a8e829: Merge branch 'master' of https://github.com/openjdk/leyden into premain 0628708c: Fixed CDS dump-time crash when running spring-petclinic cb990ebb: Merge branch 'premain' of https://github.com/openjdk/leyden into premain b3d60142: Enable StoreCachedCode/LoadCachedCode by default for new "1-step" workflow. Only String.charAt() is compiled. 878f9211: Replace UseNewCode with Xlog a5a03fcf: Reduce upcalls into Java code when preloading classes 2b430286: Fixed bug where nested init of hidden classes may cause the JIT see unloaded classes d9dfd990: Slightly reduce L2 code load failures e54e70eb: Fix aarch64 ic_stuc_code_size when SCC is used 052f03ab: Initialize classes with no <clinit> methods at class preloading time 3fb808ee: Cleaned up -Xlog:scc to print to the specified LogStream instead of tty 6457b720: disabled preresolution of invokehandle due to crashes in petclinit cddf20a5: Record and verify VM configuration 9193e092: Work-around JDK-8316802: unresolved classes may be countered by compiler 006a379f: Force some java.lang.invoke classes to be pre-initialized at dump time 2cd2c1d6: Skip not entrant preload code 397ffe78: 8316994: Avoid modifying ClassLoader and Module objects during -Xshare:dump 805509e4: Avoid archiving lambda proxies of interfaces that have clinit, as that can change program execution order. c8f40024: 8316802: unresolved classes may be encountered by compiler 751985bb: added ClassPrelinker::replay_training_at_init_for_javabase_preloaded_classes() 5fdb6150: Concurrect training replay ba3525df: Merge branch 'master' of https://github.com/openjdk/leyden into premain 7beb8afb: added SA support for TrainingReplayThread 66472a00: Add more initiated classes to preloaded dictionaries d7aaf280: 8317269: Preload all verification dependencies for archived classes fc7a5714: Fix windows build (part1) 0ddd91d7: Fix some compile errors b88c089e: CDSPreimage: avoid buffer overflow when composing command line 4c252ff1: Fixed petclinit crash with -XX:CacheDataStore f2dcff59: fixed -Xlog:cds+resolve=debug crash when dumping dynamic archive 329f7083: Added perf counter for ClassPrelinker 9fbe454b: Partial fix for minimal build failure (minimal build is still not working) 824e20a9: Temp work around for JDK-8317841: do not archive spring's dynamically generated classes d8f230d6: Improve -XX:+TraceBytecodes output e6ffcc14: ClassPrelinker: Handle LambdaMetafactory::altMetafactory BSM d5d5675a: ciInstanceKlass::print_impl() should not print to tty 73757d06: Print CodeCache contents 77d438dd: Fixed crash when perf counters were used without UsePerfData ebd5cd99: Fixed crash with spring-clinic caused by the JDK-8317841 work-around 36ec4386: Merge remote-tracking branch 'origin/premain' into premain fdbbfbe5: Trace class initialization 6be1da15: Improve logging 41c18f3f: Use :: in class::methods output e604a480: Include recompilation_schedule in -XX:+PrintSharedArchiveAndExit output 2b0a2761: Improve TrainingData validation 9d46f861: Fixed "guarantee(tag_at(which).is_klass()) failed: Corrupted constant pool" when running SpringRest.java a7b1d41d: Proper support for -XX:+RecordTraining for new workflow 5d601b75: Merge remote-tracking branch 'origin/premain' into premain 7a1ac31b: Archive all MethodCounters and MDOs w/ -XX:+ForceProfiling 3367220a: C2: full clinit barrier support df4778d6: SCC: Update stub lists a270cf73: Enable compiler directives for *::print (workaround) 5dfa2e3c: Profile VM locks usages on main application thread 9d14caf5: StressClassInitBarriers 6a7ed51c: Enable ProfileVMLocks with -Xlog:init 8f6dc80e: added jdk/test/lib/LeydenTester class for writing jtreg tests for old and new workflow 55aeef9d: fixed no-pch build 499e5ee5: added problem list for JDK-8318142 8ca4331e: too much -Xlog:scc*=trace caused IndyTestBase.java to crash; change to -Xlog:scc d40ed355: Fixed tier 1: serviceability/jvmti/RedefineClasses/RedefineDoubleDelete.java aa64bdbc: Fixed some JVMCI test failures 85a1cb67: load archived training data from preimage, so AOT code can be generated during final dump 49be0722: Populate shared code cache during terminal stage 31b2e9c8: ProfileVMCalls, ProfileRuntimeCalls, and misc minor logging enhancements a72c1abf: Support CDSAccess::get_archived_object_permanent_index() in new workflow f285fa40: Temp workaround for JDK-8318463 so we see the full error log when a build fails 0995df58: new workflow: transfer the training data from training run to final CDS image 7fe44552: fixed macos build; added more cases in appcds/leyden/ExcludedClasses.java 48369197: reformatted cdsConfig.hpp, to be upstreamed to mainline 82f357fd: Merge branch 'master' of https://github.com/openjdk/leyden into premain 8ddc8999: New workflow - partial fix for cleaning up excluded classes in CDS preimage ded30038: clean up CDS code 09c307a9: improved LeydenTester and move the old-workflow test cases to XxxOldWF.java a20e93ae: Added spring-petclinic as jtreg test daf1c352: Fixed handlig of unregistered classes in new workflow -- now they are passed from preimage to final image 2ac33fb0: Fixed crash in new workflow during AOT compilation in ciEnv::compute_init_state_for_precompiled() 0ea8bcc7: Memoize requests to ciMethod::inline_instructions_size() 04b7722e: 8317841: [premain] do not archive spring's dynamically generated classes 940808ea: fixed gcc build d124175e: Merge remote-tracking branch 'origin/premain' into premain 3d685701: cds: Archive Class::reflectionData 4868039e: ArchiveReflectionData: fix misc bugs 5f6cf611: ProfileVMLocks et al: minor fixes c3b8a2d6: Introduce per-queue locks for compiler queues ad04f75f: x86: build fixes 8d8e7ffb: UseNewCode -> PrintTrainingInfo a2fe5863: refactor CDS static dumping to prepare for writing cached code into archive 2223ded5: prototype for AOT support API in CDS 24b1f71b: Merge branch 'master' of https://github.com/openjdk/leyden into premain cf888902: Avoid calling maybe_compile_early() in CompilationPolicy::compile_if_required() 941b7395: Merge branch 'master' of https://github.com/openjdk/leyden into premain 355721f4: Fix a comment 1b6bbf7c: Merge branch 'premain' of https://github.com/openjdk/leyden into premain cae8bbcd: Added CDSConfig::preserve_all_dumptime_verification_states() to control whether classes verified by the old verifier can be archived 6956205d: Enabled -XX:+ArchiveReflectionData testing for old workflow 5b425c99: Merge branch 'master' of https://github.com/openjdk/leyden into premain 53f58b4a: Fix out of order locking assert 03e75986: profiling: allocate counters only when required 4f132c3f: perf: reserve enough space for perf counters dae99ee7: interpreter: fast class initialization checks c1b25c18: Make PrintBytecodeHistogram a diagnostic flag 2cfa3f9b: Conditionally disable cached code (per tier/level) fa117d9a: Partially lock-free compile queues 9fafb52e: misc logging cleanups de8c5f7f: Improve ProfileRuntimeCalls 28202ede: ProfileVMCalls: MethodHandleNative support 6fc2b196: ProfileRuntimeCalls: per-bytecode type stats f5c9ac2f: fixed comments d5f46a2a: Delay profiling (level 3) compilation of cold methods a323049d: Added HeapShared::initialize_default_subgraph_classes() to fix crashes related to ArchiveReflectionData b6da82f7: Need to link all classes first in HeapShared::initialize_default_subgraph_classes() 1df6ffc7: Improve asserts: Dump diagnostic info for unloaded classes b260a58d: Compiler queue statistics e6e3d998: Fix a race on Method::_code 603b991f: minor test updates 01fdf6cb: Merge remote-tracking branch 'refs/remotes/origin/premain' into premain e6e31107: Archive static field CP entries 01728040: Archive static method CP entries 5419a467: cleaned up usage of CDSConfig for new workflow c2b8b34a: Merge branch 'master' of https://github.com/openjdk/leyden into premain 6ac83f8b: Merge branch 'premain' of https://github.com/openjdk/leyden into premain a083bba5: Merge remote-tracking branch 'refs/remotes/origin/premain' into premain 0a6d86bc: Disable can_call_java restriction assert during prelinkage c874a89b: Fast clinit barrier support check for getstatic/putstatic aa4b88e7: Include CP entry tag name in diagnostic messages b1d06f4d: Archive virtual method CP entries fb3c95db: Archive method CP entries for invokespecial b0580852: Archive interface method CP entries 5fc06cbb: Archive invokehandle CP entries 2f4883a0: Improve nmethod dependency validation logging 5a7347ef: Bytecode resolution tracing 6ae7b1bc: support ArchiveReflectionData in new workflow f6fb9874: fixed comment in test/hotspot/jtreg/premain/javac_helloworld/run.sh 6a1dae6b: Delay SCCache::intialize until CDS archives are loaded 317ce587: moved the dumping of CDS archive in new workflow to after all VM initialization has finished 01070b76: Fixed bug where full module graph is not archived for new workflow 32ff6f10: Implemented -XX:+ArchiveDynamicProxies fd6e803a: new brnchmark framework for running very short running programs like javac f609b8cc: Merge branch 'master' of https://github.com/openjdk/leyden into premain ffb0c323: Validate preload code before publishing 448178f4: preimage: Don't tweak execution mode 3f419ca9: Remove CompileTrainingData::_top_method a1f8fada: Don't tweak execution mode when dumping final static archive d3cf79d2: preload: Link holder klass before preloading code 851add9e: Improve precompilation in one-step training mode ec8dd9af: Use normal thresholds for methods without TD 6586489a: Upgraded test/hotspot/jtreg/premain/spring-petclinic to 3.2.0 7faed7fc: Fix input checks in Vector API intrinsics 838db4ae: Fix C1/C2-only execution modes 614cc684: Don't archive resolved continuation intrinsic methods. c21af5eb: print timing stats in test/hotspot/jtreg/runtime/cds/appcds/applications/javac/JavacBenchApp.java 96997843: updated test/hotspot/jtreg/runtime/cds/appcds/leyden/SpringPetClinic.java to use version 3.2.0 02bc149e: updated SpringPetClinic.java jtreg test to add -Dspring.aot.enabled=true 59f97f7a: Refactored Leyden test supporting code to jdk.test.lib.cds.CDSAppTester e73de845: Factor in should_delay() 98e5def2: Merge remote-tracking branch 'origin/premain' into premain 8b33d432: Replace SCExclude with IgnorePrecompiled/DontPreload compiler directives 171b36be: 8324975: Add a timer to track thread cpu time 0f133e24: Cleanup stage 1 132e17bf: Cleanup stage 2 53ae54b2: Add utility classes to measure thread cpu time when profiling runtime 22e0cdad: Fix style 2a6bb953: Style bf07660e: Merge remote-tracking branch 'origin/master' into premain-leyden a09b607d: Fix merge 912673ad: Bump PerfDataMemorySize to 64K 8e14c78d: Improve deoptimization statistics 26287c89: -Xlog:init: Print CodeCache and SCCache statistics 752d8ff7: Print statistics on nmethod usage db5a3a48: Profile VM operations on main thread bad62d8b: CountBytecodesPerThread 74dc8e81: Improve MutexLocker profiling 6411b580: Disable thread info dumps with -Xlog:init f1b73aa8: Not-yet-loaded methods encountered in MDOs 66c2dd52: Archive class name w/ -XX:+ArchiveReflectionData 03bb7f8a: Fix build after merge 30167b6a: Cleanup MethodHandle perf counters 0adf2bfb: Fix build on linux 20c228d3: Warn about disarmed nmethods 9ffe8dbf: Guard counter usages with UsePerfData 02e1d310: Properly handle deopts on entry when gathering trap statistics 69923449: Upgrade bytecode counter to 64-bit b58e0e37: Add negative cache for built-in loaders 393f79de: Persist loader negative cache in CDS archive 361fc8b0: Positive lookup cache for built-in loaders and some cleanup 94d92c34: Improve compiler thread wakeup d5338d4e: Fix formatting 30014130: Merge remote-tracking branch 'upstream/master' into premain-upstream 24f8953d: Merge fixes 23420d8c: Prelink shared classes 073d304f: Merge branch 'premain' into premain-upstream 45dd3f92: Merge remote-tracking branch 'upstream/master' into premain-upstream 62a344e2: Fixed #include problems for PerfTickCounters::elapsed_counter_value_ms(), etc 33a9ad6e: Comment out the assert for safepoint check when iterating classes in DumpTimeSharedClassTable 2fc639a0: Merge branch 'premain' into premain-upstream a672b4c6: Fix x86 build 68a9c37e: Create a new DumpTimeSharedClassTable iterator for the classes loaded by builtin loaders 13ba0651: Fix a bug in writing contents of loader's negative lookup cache f1d6e862: Automate warmup testing using springboot-petclinic 23baf7f1: Update comments in WarmupMakefile b99380d4: Patch file for springboot petclinic application used by WarmupMakefile 5ce5a63d: Refactored code for deciding what classes can be pre-inited 3cdb6324: Fixed test failure in ArchiveHeapTestClass.java d67819c6: 8318064: [premain] CDS regenerated LambdaForm holder classes miss some methods 240a6782: jmeter test plan used by WarmupMakefile for springboot-petclinic 50ea3bb5: Minor improvements to WarmupMakefile e22e1cb3: Trace method invocation from interpreter c3984a7c: relocInfo et al tracing improvements e2d3cd2b: fix Windows build failure 1303cece: Merge remote-tracking branch 'origin/premain' into premain 69027b63: Fix allocation types 5012dd1b: Remove init tracking 4e9483db: Improve CompileTrainingData::_init_deps_left handling 0a5d7374: Remove TD factories for symbols 8461779d: Remove forgotten trap 29f450bf: Clear the pending_queue_processed flag on Method when it is removed from 18b0133d: Remove hard-coded path to JMeter binary 13505f2c: Remove unnecessary CTD::make() 9e34721c: Control tracing of thread time in Perf counters using a command line 8045ce6e: Fix a bug to set initial value of PerfCounter to 0 bcb9a07d: Regenerate lambda form invoker classes at the end of preimage run. This reduces the number of LambdaForm classes in final image. ade0acb9: -Xlog:scc+codecache: Print archive code info on exit 89e5c3b0: Detailed statistics on deoptimization events 42e27ba6: Rework compiler init dependency tracking 30374ac3: Improve TrainingData verification c3436de3: Improve constant pool preresolution 1ec8eb15: Archive array classes 93f1bb69: SSCache cleanups 3e7e1ecb: Code precompilation support 2cc64803: Avoid code archiving failures due to code buffer resizing on AArch64 7cbabcf3: precompilation: Improve class init state computation in CI a4d97992: Disable PreloadSharedClasses by default (same as JEP) 7afec36f: Merge branch 'master' of https://github.com/openjdk/leyden into premain 11038ecc: Reverted unnecessary -XX:-PreloadSharedClasses changes to CDS test cases de041670: fixed some minor jtreg test failures bc34228f: UsePermanentHeapObjects should be enabled for static dump only if PreloadSharedClasses==true. a2b89415: remove TestResolvedJavaMethod.java test from ProblemList-premain.txt c1344d31: Use non-throwing metaspace allocation for training data 439fe8f1: Merge branch 'premain' of github.com:openjdk/leyden into premain 4ffe4f8a: split classPreloader.cpp out of classPrelinker.cpp 83a96356: split classPreloader.cpp out of classPrelinker.cpp (step2) b25ef0a5: Added "make mainline_vs_premain" to get simple comparisons of pet clinic 9ace2f21: specify all CDS optimizations explicitly in pet clinic makefile de0cf7b9: Added "make compare_premain_builds" and "make diff_training_logs" for pet-clinic d9366c89: Fix handling of array class (ObjArrayKlass) of TypeArrayKlass e17d2f5a: When -XX:+PreloadSharedClasses is given, enable all CDS optimizations that depends on it bd80f5b8: Fixed "Booted and returned" timing in spring-petclinic to also include time spent in the JVM before main() method is executed b091912a: 8328563: Store java.lang.ClassLoader::{packages, package2certs} into CDS archive 76adcbf1: Merge branch 'master' of https://github.com/openjdk/leyden into premain b31eddf5: Save and relocate stop msg strings in cached code on aarch64 85c0eea0: Remove InstanceKlass::_training_data. c93abe3f: Fixed macosx/aarch64 build dab044f9: Do not write @cp for classes that are regenerated during the JDK build 2003a807: Simplified archiving of Universe:*_exception_instance() objects e2902a0e: Quick preloading of classes for the boot loader without calling SystemDictionary::load_instance_class() 6c0c6ff3: Cleaned up ClassLinker APIs for pre-resolution of ConstantPool entries b298683e: Merge branch 'master' of https://github.com/openjdk/leyden into premain a81c1cc9: Fixed crashing with https://github.com/sdeleuze/petclinic-efficient-container 5c1ad75f: Merge branch 'master' into premain b3933d8b: more clean up of pre-resolution of CP entries b2bc2555: 8329721: Store java.lang.Boolean.{TRUE,FALSE} in CDS archive 0217a396: 8329728: Read arbitrarily long lines in ClassListParser aa6d5444: Minor fixes to WarmupMakefile 7ecd661f: Cleanup remove_unshareable_info() and restore_unshareable_info() 908503f1: (1) enable ArchiveLoaderLookupCache, by default when PreloadSharedClasses is enabled; (2) do not disable Inline for new workflow 057360f9: Add targets to run warmup test using new single-step workflow 338719d4: Refactoring: moved class init code into cds/classPreinitializer.cpp f546e292: More refactoring: move code related to pre-initialization to classPreinitializer.cpp fdcc1180: Refactor: move is_vm_class() into ClassPreloader; fixed inconsistent use of xxx_klass vs _xxx_class in ClassPreloader/ClassPrelinker e211d5d7: Moved the management of the "receipes" to the FinalImageRecipes class 194ae8ad: Added build/test scripts for new demo: https://helidon.io/docs/v4/se/guides/quickstart a98bb060: Add missing reference traversal 4b1f9091: Remove chaining of CTDs in a list. Capture only the ones we need. 5f276f58: Remove unused allocate() functions 14902cb2: Added build/test scripts for new demo: https://quarkus.io/guides/getting-started c9feeb6f: Merge branch 'master' of https://github.com/openjdk/leyden into premain 2e3f59b9: Allocate all TD in the null class loader data 48d45275: Fix warning 811bad88: 8321944: fix PcDesc cache initialization 64987121: Original code unpacked from https://guides.micronaut.io/latest/creating-your-first-micronaut-app-maven-… , downloaded on 2024/04/22 9fa97221: Added "Micronaut First App" as a test case 04a95521: Added "make bench" in all four framework demos; added relative performance numbers to premain/README.md c40b2993: Fixed graphs in MD file 74c0bfad: minor benchmark tweaks 8571089b: 8331086: TrainingData allocation from Metaspace may fail 5a39b00d: Merge branch 'master' of https://github.com/openjdk/leyden into premain 192edefb: set CachedCodeMaxSize to 512M when StoreCachedCode is enabled aa66815f: Improved remove_unshareable_info for TrainingData (to avoid race condition where compiler thread might be adding more TD when we are collecting them ourside of the CDS safepoint) 039c0219: Added instructions for running demos 12ab8469: Improved benchmarks docs; added "make compare_premain_builds" ba07d82a: Merge branch 'premain' of https://github.com/openjdk/leyden into premain 674deded: Copy contents of ProblemList-premain.txt to ProblemList.txt, in preparation of regular tiers1-3 testing 9076b74d: Disable CDS if runtime VM options are incompatible with archives dumped with -XX:+PreloadSharedClasses d44cdcbf: No need to call TrainingData::restore_all_unshareable_info 9e106c59: Added instructions for running Leyden-specific jtreg tests eb25dca8: Avoid grabbing locks inside MethodData::extra_data_lock aee254fa: Merge branch 'master' into premain ba8bf2e7: Merge branch 'master' of https://github.com/openjdk/leyden into premain 49baa911: Fixed typo in test/hotspot/jtreg/premain/lib/build-for-jtreg.sh 84afb72e: Remove restore_unshareable_info() 86ec27a6: Fixed _compare_premain_builds in test/hotspot/jtreg/premain/lib/Bench.gmk to be compatible with premain/spring-petclinic/Makefile f11f4604: support benchmarking for new workflow 9c7ef4ea: include missing TASKSET arg b2e204aa: Refactored applications/JavacBench.java test; added sanity validation 3674fc32: Clean up ClassListParser 06232a3c: prepare for upstreaming JDK-8293980 cd261d16: Merge branch 'master' of https://github.com/openjdk/leyden into premain d85a7e89: fixed merge b8b686a2: make/data/hotspot-symbols/symbols-unix is no longer needed (removed in mainline) 15f51139: Add snapshotting mechanism for training data 8dee825a: Moved Leyden overview docs to $REPO/README.md 32811147: Fixed typo and added disclaimers 4faa7202: Added checks for G1GC during new workflow training run f7d6f17a: Removed Leyden-specific workaround in test cases 7e9d14f0: 8332340: Add JavacBench as a test case for CDS c67940c7: 8333222: Allow SerialGC and ParallelGC to be used in Leyden training run 9c138a8e: Convert not fatal log_warning(scc) to log_info(scc) 739a39bf: (a) InterpreterRuntime::cds_resolve_invoke does not need mh param; (b) merged some changes from https://github.com/openjdk/jdk/pull/19355 ada67450: (1) Added UseCompatibleCompressedOops flag to allow AOT code to work for all heap ranges; (2) updated README.md for "same collector" limitation 485dee94: Enable -Xshare:on by default for new workflow -- so people know immediately if the Leyden optimizations are disabled due to a misconfiguration 9834890b: Fixed failure in ExcludedClassesOldWF.java caused by the previous commit 5214ff1e: Fixed -XX:+UseCompatibleCompressedOops crash on macos 789f1452: Added notes about supported garbage collectors 74171dd1: Treat SCC warnings as error when -Xshare:on 16e4da7f: Disabled tests for differnt coop encodings 0518a5d2: Proper handling of pending stale tasks in CompileQueue abdbf473: Editorial changes to README, plus better mermaid format from lujaniuk d0981826: Merge branch 'master' of https://github.com/openjdk/leyden into premain 5d856130: small clean up in README.md e4c038df: Clarify that this repo contains only code from the "premain" prototype 5d3441c4: Hand-merged some changes from JDK-8293980 (in preparation for mainline merge) 29503891: Merge commit 'eb2488fd1781af49d936348d5f75731de2006ce7' into premain d9e797b3: reduce the use of is_on; add missing class linking statistics 8dc7f15d: Merge branch 'master' of https://github.com/openjdk/leyden into premain 8f2eb149: Factor out recompilation logic e2b46a20: Merge branch 'premain' of github.com:openjdk/leyden into premain ce790adc: remove leftover commented code during last merge 173181cc: Added test case for 8317269: [premain] Archive old classes in verified state when CDSLoadedClasses is enabled ce3ce7aa: Removed incorrect message "Cannot dump shared archive while using shared archive" d415c65b: Instead of using ArchiveInvokeDynamic directly, use CDSConfig::is_dumping_invokedynamic() 8716f47e: Merge branch 'premain' of github.com:openjdk/leyden into premain 2bb2fbc4: Always set TD::_holder. 5ebf8d6c: Stabilize find() behavior during snapshot 3726f1df: Cleanup 159c6693: Simplify KTD constructor dc69d066: 8334421: assert(!oldbox->is_unbalanced()) failed: this should not be called for unbalanced region 5b27856a: 8335220: C2: Missing check for Opaque4 node in EscapeAnalysis c27385e3: 8335221: Some C2 intrinsics incorrectly assume that type argument is compile-time constant 517e4b23: Merge branch 'premain-ea' of https://github.com/openjdk/leyden into premain 098c309d: 8335492: Some hidden classes are not archived for ArchiveInvokeDynamic bd601c50: 8309634: Resolve CONSTANT_MethodRef at CDS dump time d2446866: fixed windows build 40373c0a: Merge branch 'master' of https://github.com/openjdk/leyden into premain 43e77c8e: Added test case for 8328563: Store java.lang.ClassLoader::{packages, package2certs} into CDS archive d9e3e5fe: 8335804: ArchiveInvokeDynamic causes BootstrapMethodError 77e9f24c: Print more info about command-line to help debugging the dumping process c49bd56b: Guard JVMTI related code with INCLUDE_JVMTI 5fdfef1d: Added more test case for JDK-8317269: Archive old classes in verified state when CDSLoadedClasses is enabled dbdccb37: 8335790: Store protection domains into CDS archive f65cccf9: 8336033: [premain] assert(comp != nullptr) when using SCCache with -XX:TieredStopAtLevel=1 5545aa0c: Add option to omit archive heap verification 79329d86: Resolve merge conflicts 77eff765: Added Pet Clinic benchmark to track performance of JEP-8315737 ded7dae0: Work-around JDK-8336414 which causes failure in ResolvedConstants.java f519a8a9: Removed incorrect message "Cannot dump shared archive while using shared archive" c2a5c9ac: 8335804: ArchiveInvokeDynamic causes BootstrapMethodError 4c385b55: 8336033: [premain] assert(comp != nullptr) when using SCCache with -XX:TieredStopAtLevel=1 d47557ad: Cache few stubs on aarch64 similar to x64 to avoid empty cached code 1b665a99: 8336144: assert(CDSConfig::is_dumping_invokedynamic()) when ZGC is used during archive creation 7ae716cf: Merge branch 'premain-ea' of https://github.com/openjdk/leyden into premain b949235f: Add missing far branch checks on aarch64 to generate trampolines 34484563: Fixed CDS failures in tier3 (-XX:+UseSerialGC, -XX:+UseZGC, etc) f1c370f9: More fixes for CDS failures in tier3 (-XX:+UseZGC, etc) baad503c: 8336562: Quit CacheDataStore creation when incompatible module options are specified 35a48278: Merge branch 'master' of https://github.com/openjdk/leyden into premain 974f53b6: SCC: Handle ConstantDynamic CP entries ed32488c: C2: Fix upcall into compiler runtime w/ -XX:ClassInitBarrierMode=2 a03a63e3: Merge branch 'premain' of https://github.com/openjdk/leyden into premain 8423515c: Change TD has table to have keys with direct references ec5eb996: Clean up cdsEnumKlass.cpp code for upstreaming to mainline b24e6fa2: New "AOT" options (-XX:AOTCache=xxx, etc) implemented as aliases to existing CDS options 37b332cc: fixed typo f972b8dc: Added -XX:AOTMode as specified in latest JEP draft. Removed RecordAOTConfiguration/CreateAOTCache c0826c59: Fixed silly mistake in last commit d7c97f22: Added spring-boot-getting-started demo 1e18a09d: Refactoring of ClassPreloader and ClassPrelinker. Now we have AOTClassLinker, AOTConstantPoolResolver and AOTLinkedClassBulkLoader ee11cf1b: Exclude test cases that cannot be executed with -Dtest.dynamic.cds.archive=true 12225130: Added -XX:AOTMode=on/off/auto dce3658c: Cleanup: added CDSConfig::is_using_aot_linked_classes() 9a4ea6ff: preload classes tests do not work with ZGC d9811b4a: Do not use decomp count for one step workflow c01a9bce: fixed windows build 8b148497: renamed -XX:+PreloadSharedClasses to -XX:+AOTClassLinking 0d054eb4: 8337965: Add null check to ClaimMetadataVisitingOopIterateClosure::do_cld() 0bcbbff9: more clean up on new AOTxxx command-line flags 82c7ad0d: 8338011: CDS archived heap object support for 64-bit Windows 9f24a918: Ignore decomp counter for one-step. Cleanup in prints. 65bd17a2: More AOTClassLinking clean up 5ca207db: Merge branch 'master' of https://github.com/openjdk/leyden into premain b065ce74: Set _sc_count correctly when only one of c1 compiler or c2 compiler is enabled e4059458: Enable finding scc entry when _sc_count is 0 (using -XX:-UseCodeLoadThread) 2158a6a5: jcheck configuration for Leyden 864228c7: Update JEP titles, and add a general warning that flags will be renamed a5daf90b: Fixed runtime/cds/DeterministicDump.java failure on Windows 1ee287c0: (1) Fixed Windows failures; (2) Clean up use of CDSConfig::is_xxx(); (3) Consolidate SpringPetClinic.java tests 7e207bd7: Merged ProblemList-premain.txt into ProblemList.txt cb30dfbb: Renamed -XX:+ArchiveInvokeDynamic to -XX:+AOTInvokeDynamicLinking, and changed to a diagnostic flag 641fdabf: More clean up of -XX:+AOTInvokeDynamicLinking before upstreaming to mainline c319a3ee: AOT G1 barrier loads region grain and card shifts via runtime constants data area 06f6f57b: Merge branch 'master' of https://github.com/openjdk/leyden into premain 2d15c193: temp work-around to get premain to work with JDK-8338532 and JDK-8336856; todo -- fix this more properly 126461c0: Enable 1-step workflow with Epsilon GC a371fddb: Merge branch 'premain' of https://github.com/openjdk/leyden into premain 7f633b95: Keep cached strings longer. Fix ExternalsRecorder::print_statistics() cc111760: Cleanup AOT runtime constants code b96c5113: Fix various build configs for clean PRs and happy CIs 1bd62091: Simplified the archiving of enum class mirrors 3a84df9d: Renamed classPreinitializer.cpp to aotClassInitializer.cpp; more heapShared.cpp clean up e33ecb3e: Simplified dumping of MethodType with the AOTHolder pattern 9bab6122: Experimental feature to allow special instrumentation code to be executed during training run. See CacheOnlyClassesIn.java for details 242e33d3: More clean up for upstreaming JDK-8329706 Implement -XX:+AOTClassLinking cf036d70: Fix SCCache closing races d7bd0ccc: Rework rich PrintCompilation logs 678b50d1: Prefer to take AOT method from the SC queue faster 5fd814c0: Improve incapsulation d23b9f2d: Mark Leyden tests that require artifacts with "external-dep" 07ad2086: Tidy up a8b8d7da: Reformat internal profiling stats 77811091: Method name AOTClassInitializer::has_non_default_static_fields should be has_default_static_fields 6c83df44: 8340559: [premain] java process crash with -XX:+PrintTrainingInfo 7a6fadca: 8340869: [premain] NullPointerException with LambdaWithUseImplMethodHandle.java ac3f40d0: 8340836: Method.invoke fails with java.lang.AssertionError with AOTClassLinking and system assertions enabled 21d3ed73: Merged some code from upstreaming PRs of JEP 483 9cf06243: Added some CDSHeapVerifier rules for ArchiveLoaderLookupCache 5bd0b880: Sync some files with latest JEP 483 PRs 173fb74c: 8339020: Remove unused HeapShared::calculate_oopmap 618907c4: 8339351: Remove duplicate line in FileMapHeader::print e9521b21: 8338912: CDS: Segmented roots array 17571b6d: Merge branch 'master' into premain cc1d328d: Merge branch 'master' into premain ce1b43aa: Need to clean up _orig_to_scratch_object_table when classes are unloaded (e.g., when jdk.internal.event.Event gets rewritten by JFR) a3a64c76: Merge branch 'master' into premain 7b72b048: FileMapInfo::write_bitmap_region bug after premain merge b187389b: Merge branch 'master' into premain ab9d91e5: Merge branch 'master' into premain d504df0e: fix g1 barrier register use after late expand 5029a175: Ensure branch target is correctly relocated 88b6b4b2: further correction to register use in g1 barriers 4e0412e6: Enable 1-step workflow with Shenandoah GC a1672ea6: Merge branch 'premain' of https://github.com/openjdk/leyden into premain 21640b10: Make TD::allocate() a template to reduce boilerplate 52eee916: fix -Xlog:init output 1c6e576c: Cleanup includes cd8c4f40: Remove MethodInfoDictionary 8ebb8318: Merge branch 'master' into premain 163cae69: Fixed merge f4e1f040: Merge branch jdk/master into premain e4017de3: Prepare to merge with 41a2d49f0a1ed298b8ab023ce634335464454fe7 - hand merge some files cb8c7904: Merge remote-tracking branch 'jdk/master' into premain b5459e0a: Fixed merge ac533a7e: Fixed CDSAccess::get_archived_object_permanent_index() to only keep alive archived oops that have a permanent index (previously every archived oop was kept alive) c2871bb1: Merge branch 'premain' of https://github.com/openjdk/leyden into premain b3e5a418: Remove unnecessary logs for CDSAccess::get_archived_object_permanent_index()

1 0

AOT caching of WeakReference
by ioi.lam＠oracle.com 19 Nov '24

19 Nov '24

Here's my experiment with storing weak references in the AOT cache https://github.com/iklam/jdk/blob/8341587-example-of-weakref-problem-in-aot… As discussed in the Leyden meeting today, there seem to be at least two issues - In premain, CDS keeps every archived heap object alive (look for UsePermanentHeapObjects in the code). I plan to get rid of this shortly. UsePermanentHeapObjects is intended for the AOT compiler to access heap objects, but it should be implemented differently (only remember objects that are required by AOT code, which should include only Strings and class mirrors). - WeakReference.queue is not pointing to the production-time value of ReferenceQueue.NULL Thanks - Ioi

3 4

git: openjdk/leyden: premain: Fixed CDSAccess::get_archived_object_permanent_index() to only keep alive archived oops that have a permanent index (previously every archived oop was kept alive)
by duke 19 Nov '24

19 Nov '24

Changeset: ac533a7e Branch: premain Author: iklam <ilam3388(a)gmail.com> Date: 2024-11-18 17:20:45 +0000 URL: https://git.openjdk.org/leyden/commit/ac533a7e29810400fd6ccbbeab01e478dfd2f… Fixed CDSAccess::get_archived_object_permanent_index() to only keep alive archived oops that have a permanent index (previously every archived oop was kept alive) ! src/hotspot/share/cds/archiveBuilder.cpp ! src/hotspot/share/cds/archiveHeapLoader.cpp ! src/hotspot/share/cds/archiveHeapLoader.hpp ! src/hotspot/share/cds/archiveHeapWriter.cpp ! src/hotspot/share/cds/filemap.cpp ! src/hotspot/share/cds/heapShared.cpp ! src/hotspot/share/cds/heapShared.hpp ! src/hotspot/share/cds/metaspaceShared.cpp ! src/hotspot/share/code/SCCache.cpp

1 0

JEP 483 targeted to upcoming JDK 24 release
by John Rose 14 Nov '24

14 Nov '24

Our first “premain” feature, affectionally called “JEP One”, is now targeted[1]. [1] https://mail.openjdk.org/pipermail/jdk-dev/2024-November/009645.html Congratulations to everyone who helped reach this important milestone! JEP One will provide a firm foundation for a number of interesting future JEPs, as seen in our EA. I think Ioi will be preparing a final version of the PR, which we will re-review and re-test quickly, and then push during the next day or two. The conflict level is small (just one conflict with Lilliput) so we are hopeful the rebase will work smoothly. There is some technical debt here to follow up on. We will be fixing a small handful of simple JCK failures which we know about, and any other bugs that crop up at at level P3 or higher. As a bigger step, we will be simplifying some parts of the changes for JEP One. Many simplifications will probably land in JDK 25. For example, workarounds avoiding SoftReference usage will be removed after we get the kinks out of using SoftReference in the assembly phase (AP). And the AP heap walking logic can be made simpler and more robust. There is a long list of possible cleanups in JDK-8343023[2]. [2] https://bugs.openjdk.org/browse/JDK-8343023 Our big user visible items, probably follow-up JEPs, will include a simplified workflow (make the AOT config generation transparent, optionally), persisting of method profiles from the training run (TR), and persisting of compiled method code (the “JIT cache”). All of these are visible in the Early Access builds, and were demonstrated last year at JVMLS 2023. As an independent workflow RFE, I want to get Mat Carter’s end-of-training hook pushed whenever we think it is ready. That is https://bugs.openjdk.org/browse/JDK-8335358. It is useful whether the workflow as a whole is improved or kept as it is today. I am also hoping to clean up and rationalize the boot sequence of the JVM, the initialization of the First Hundred classes. (They are about a hundred, not exactly 100.) This is likely to involve more explicit declarations of ordering conditions inside the JDK source code itself. We also want to split one class C into two CA+CR so that C can be initialized at assembly time and CR can be initialized at production runtime. Instead of CR, we might also double down on runtime-setup methods in CA, but that precludes the use of final fields in CA, since finals in CA would be initialized by the assembly phase, and not revisited at startup. We are avoiding magic that makes final variables change, so if a variable needs to change, it cannot be a final field. It can be stable, though; we expect to amend the contract of stable variables to allow changes during the assembly phase. Thanks, — John

1 0

git: openjdk/leyden: premain: Remove MethodInfoDictionary
by duke 13 Nov '24

13 Nov '24

Changeset: cd8c4f40 Branch: premain Author: Igor Veresov <igor.veresov(a)oracle.com> Date: 2024-11-13 15:32:38 +0000 URL: https://git.openjdk.org/leyden/commit/cd8c4f4002eb435d8ea8db5c70e9b9b1de7e4… Remove MethodInfoDictionary ! src/hotspot/share/cds/archiveBuilder.cpp ! src/hotspot/share/cds/dynamicArchive.cpp ! src/hotspot/share/cds/metaspaceShared.cpp - src/hotspot/share/cds/methodDataDictionary.cpp - src/hotspot/share/cds/methodDataDictionary.hpp ! src/hotspot/share/classfile/systemDictionaryShared.cpp ! src/hotspot/share/classfile/systemDictionaryShared.hpp ! src/hotspot/share/oops/trainingData.hpp

1 0

git: openjdk/leyden: premain: Cleanup includes
by duke 13 Nov '24

13 Nov '24

Changeset: 1c6e576c Branch: premain Author: Igor Veresov <igor.veresov(a)oracle.com> Date: 2024-11-12 19:34:02 +0000 URL: https://git.openjdk.org/leyden/commit/1c6e576cb55c2da118afdfda6a450259bfe4e… Cleanup includes ! src/hotspot/share/cds/aotClassInitializer.cpp ! src/hotspot/share/cds/methodDataDictionary.hpp ! src/hotspot/share/compiler/precompiler.cpp ! src/hotspot/share/oops/recompilationSchedule.cpp ! src/hotspot/share/oops/trainingData.cpp ! src/hotspot/share/oops/trainingData.hpp

1 0

git: openjdk/leyden: created branch 8341587-example-of-weakref-problem-in-aot-cache based on the branch premain containing 1 unique commit
by duke 12 Nov '24

12 Nov '24

The following commits are unique to the 8341587-example-of-weakref-problem-in-aot-cache branch: ======================================================== 4730652e: An example of WeakReference not working when stored inside the AOT cache

1 0

git: openjdk/leyden: hermetic-java-runtime: Make error() a static function in childproc.c. This is to resolve `ld: error: duplicate symbol: error` issue on hermetic Java with libjava.a and third party library.
by duke 11 Nov '24

11 Nov '24

Changeset: e20a9155 Branch: hermetic-java-runtime Author: Jiangli Zhou <jianglizhou(a)google.com> Date: 2024-11-11 14:14:56 +0000 URL: https://git.openjdk.org/leyden/commit/e20a9155ae37caa5151005997fc9b4478bb20… Make error() a static function in childproc.c. This is to resolve `ld: error: duplicate symbol: error` issue on hermetic Java with libjava.a and third party library. ! src/java.base/unix/native/libjava/childproc.c

1 0

git: openjdk/leyden: premain: fix -Xlog:init output
by duke 07 Nov '24

07 Nov '24

Changeset: 52eee916 Branch: premain Author: Vladimir Ivanov <vladimir.x.ivanov(a)oracle.com> Date: 2024-11-07 15:53:09 +0000 URL: https://git.openjdk.org/leyden/commit/52eee916777c6b7a7a70ea6f55750935e0033… fix -Xlog:init output ! src/hotspot/share/cds/aotLinkedClassBulkLoader.cpp ! src/hotspot/share/cds/aotLinkedClassBulkLoader.hpp ! src/hotspot/share/classfile/classLoader.cpp ! src/hotspot/share/runtime/java.cpp ! src/hotspot/share/runtime/java.hpp ! src/hotspot/share/runtime/sharedRuntime.cpp ! src/hotspot/share/runtime/threads.cpp

1 0