RFR: 8352092: -XX:AOTMode=record crashes with InstanceKlass in allocated state

[David Holmes]

On Sat, 22 Mar 2025 04:39:40 GMT, Ioi Lam <iklam at openjdk.org> wrote:

> Please review this small fix for a crash that can be caused only with JNI `DefineClass()` calls, which can leave a class in the `allocated` state inside a `ClassLoaderData` when the class is in a prohibited package such as `java/foo`.
> 
> The bug was found when running the JCK with an AOT cache. I tried to write a jtreg regression test but it's not possible to use pure Java code to reproduce this condition -- attempts by Java code to define classes in a prohibited package will be blocked from entering the ClassFileParser.

I can't convince myself this is the right place to fix this.

If we attempt to use `DefineClass` for a class in an illegal package, it will fail won't it? In which case how do we end up with any `instanceKlass` object to archive? Shouldn't it be  deleted, assuming it ever got created? And if it was created do we really want to archive classes that are not even loaded?

-------------

PR Review: https://git.openjdk.org/jdk/pull/24172#pullrequestreview-2709280437