RFR: 8276126: Dump time class transformation causes heap objects of non-boot classes to be archived [v4]

[Yumin Qi]

On Tue, 30 Nov 2021 17:51:46 GMT, Calvin Cheung <ccheung at openjdk.org> wrote:

>> During CDS static dump, if a class loaded by a boot class loader is transformed by a java agent, the transformed class is mistakenly identified as a class loaded by a custom loader. This would cause archiving of the heap objects to fail.
>> 
>> The proposed patch corrects the identification of the transformed class and will not include it in the archive.
>> Also, if a transformed class is detected during dump time, the archiving of the heap objects will be disabled.
>> 
>> Testing: Oracle CI tiers 1-3, tier4 in progress.
>
> Calvin Cheung has updated the pull request incrementally with one additional commit since the last revision:
> 
>   remove the unused stream parameter in ClassLoaderExt::record_result()

src/hotspot/share/classfile/classLoaderExt.cpp line 250:

> 248:   result->set_shared_class_loader_type(classloader_type);
> 249: #if INCLUDE_CDS_JAVA_HEAP
> 250:   if (DumpSharedSpaces && AllowArchivingWithJavaAgent && classloader_type == ClassLoader::BOOT_LOADER &&

the function is call during dump time (the assert at first line of this function check that) so DumpSharedSpace can be removed.
I am a little confused by this part:  If 'redefined' is 'true', The 'redefined' only should be enough for set disable_writing() whether other conditions stands or not?

-------------

PR: https://git.openjdk.java.net/jdk/pull/6484