RFR: 8293336: AOT-linking of invokedynamic for lambda expression and string concat [v5]

[Ioi Lam]

On Tue, 15 Oct 2024 19:38:43 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> Ioi Lam has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 29 commits:
>> 
>>  - @DanHeidinga comments -- added ConcurrentHashMap::runtimeSetup() to init NCPU to runtime value; also use the same runtimeSetup() pattern to call registerNatives() for Class.java and Unsafe.java
>>  - Merge branch 'jep-483-step-06-8311071-avoid-soft-refs-in-java-lang-invoke' into jep-483-step-07-8293336-store-lambda-forms-in-cds-archive
>>  - Fixed JDK-8341988: jstack launched with AOT cache created with -XX:+AOTClassLinking crashes
>>  - 8341600: [premain] Automatic aot-init of classes used by java.lang.invoke
>>  - @adinn comments
>>  - improve checks for not changing <clinit> order for aot linking of lambda; added comprehensive test cases: AOTLinkedLambdasApp::testClinitOrder()
>>  - Clean up of aotClassInitializer and cdsHeaVerifier; added lambda test cases for <clinit> order of app classes
>>  - Merge branch 'jep-483-step-06-8311071-avoid-soft-refs-in-java-lang-invoke' into jep-483-step-07-8293336-store-lambda-forms-in-cds-archive
>>  - Require all <clinit> of supertypes of aot-inited classes to be executed in assembly phase
>>  - Limit the use of AOTHolder
>>  - ... and 19 more: https://git.openjdk.org/jdk/compare/e46b910a...382446d4
>
> src/hotspot/share/memory/iterator.inline.hpp line 53:
> 
>> 51:     // class has not yet been loaded by CDS.
>> 52:     cld->oops_do(this, _claim);
>> 53:   }
> 
> Could you show the stack trace for this? I want to understand better why we are iterating over objects that don't have a loaded class.

Here's the call stack if I remove the "if" check on line 49. It can be reproduced with test/hotspot/jtreg/runtime/cds/appcds/aotClassLinking/BulkLoaderTest.java from the PR repo.


10 javaSignalHandler (sig=11, info=0x7ffff428fdb0, context=0x7ffff428fc80)
12 Atomic::PlatformLoad::operator() (this=0x7ffff429097f, dest=0x28)
13 Atomic::LoadImpl, void>::operator() (this=0x7ffff42909af, dest=0x28)
14 Atomic::load (dest=0x28)
15 ClassLoaderData::try_claim (this=0x0, claim=4)
16 ClassLoaderData::oops_do (this=0x0, f=0x7fff6c014c88, claim_value=4, clear_mod_oops=false)
17 ClaimMetadataVisitingOopIterateClosure::do_cld (this=0x7fff6c014c88, cld=0x0)
18 ClaimMetadataVisitingOopIterateClosure::do_klass (this=0x7fff6c014c88, k=0x7fff763655d8)
19 call_do_klass (closure=0x7fff6c014c88, k=0x7fff763655d8)
20 Devirtualizer::do_klass (closure=0x7fff6c014c88, k=0x7fff763655d8)
21 InstanceKlass::oop_oop_iterate (closure=0x7fff6c014c88, obj=0xfffd72a0, this=0x7fff763655d8)
22 OopOopIterateDispatch::Table::oop_oop_iterate (cl=0x7fff6c014c88, obj=0xfffd72a0, k=0x7fff763655d8)
23 OopIteratorClosureDispatch::oop_oop_iterate (cl=0x7fff6c014c88, obj=0xfffd72a0, klass=0x7fff763655d8)
24 oopDesc::oop_iterate (this=0xfffd72a0, cl=0x7fff6c014c88)
25 G1FullGCMarker::follow_object (this=0x7fff6c014630, obj=0xfffd72a0)
26 G1FullGCMarker::publish_and_drain_oop_tasks (this=0x7fff6c014630)
27 G1FullGCMarker::follow_marking_stacks (this=0x7fff6c014630)
28 G1FullGCMarker::complete_marking (this=0x7fff6c014630, oop_stacks=0x7fffd445c6b8, array_stacks=0x7fffd445c6d0, terminator=0x7fffd445c190)
29 G1FullGCMarkTask::work (this=0x7fffd445c030, worker_id=0)
30 WorkerTaskDispatcher::worker_run_task (this=0x7ffff008b9b8)
31 WorkerThread::run (this=0x7ffff008bac0)


This is in a GC triggered by here (with -XX:VerifyArchivedFields=2).


void HeapShared::init_archived_fields_for(Klass* k, 
     const ArchivedKlassSubGraphInfoRecord* record) {
  verify_the_heap(k, "before");


At frame 18:


#18 0x00007ffff58505d1 in ClaimMetadataVisitingOopIterateClosure::do_klass 
    (this=0x7fff6c014c88, k=0x7fff763655d8)
    at src/hotspot/share/memory/iterator.inline.hpp:58
58	  ClaimMetadataVisitingOopIterateClosure::do_cld(cld);
(gdb) psym k->_name
$1 = 0x7fff765430be "BulkLoaderTestApp$$Lambda+0x800000013"
(gdb) p k->_class_loader_data
$2 = (ClassLoaderData *) 0x0
(gdb) p SystemDictionary::_java_system_loader 
$3 = {_obj = 0x0}


At frame 21:


#21 0x00007ffff6132f6f in InstanceKlass::oop_oop_iterate<narrowOop, G1MarkAndPushClosure> 
    (closure=0x7fff6c014c88, obj=0xfffd72a0, this=0x7fff763655d8)
    at src/hotspot/share/oops/instanceKlass.inline.hpp:164
164	    Devirtualizer::do_klass(closure, this);
(gdb) call pp(obj)
"Executing pp"
BulkLoaderTestApp$$Lambda/0x800000013 
{0x00000000fffd72a0} - klass: 'BulkLoaderTestApp$$Lambda+0x800000013' - flags: is_hidden_class 


The root cause is:

- A lambda has been aot-resolved for a class loaded by the app loader
- There is an instance (`0xfffd72a0`) of the lambda proxy class (`BulkLoaderTestApp$$Lambda+0x800000013`) that represents this call site 
- At the point of failure, all archived heap objects, including `0xfffd72a0`, are reachable (through `HeapShared::_root_segments`).
- However, we are still early in VM bootstrap. The `_java_system_loader` isn't available yet, so we are not yet able to load the lambda proxy class.

Before JEP 483, all archived objects are from java.base, and the `VerifyArchivedFields` code was careful to not trigger a GC until `is_init_completed()` becomes true.

However, with JEP 483, it's possible to have an archived object from the app class loader, and it's legally possible for a GC to happen before the app loader is available. Thus we come into this situation.

(Ping @fisk @coleenp in case you're curious)

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/21143#discussion_r1802291536