RFR: 8356693: AOT assembly phase fails with -javaagent

[Ashutosh Mehra]

On Sun, 11 May 2025 04:57:44 GMT, Ioi Lam <iklam at openjdk.org> wrote:

> https://openjdk.org/jeps/483 mentions:
> 
>> To enjoy the benefits of the AOT cache generated during a training run, the training run and all subsequent runs must be essentially similar. [...] All runs must not use JVMTI agents that can arbitrarily rewrite classfiles  using ClassFileLoadHook.
> 
> However, when *any* java agent is specified in the training run, the JVM fails at start-up. E.g.,
> 
> 
> $ java -XX:AOTMode=record -javaagent:agent.jar -cp app.jar App
> Error occurred during CDS dumping
> Must enable AllowArchivingWithJavaAgent in order to run Java agent during CDS dumping
> 
> 
> With the AOT cache, the main concern for JVMTI agents is that they can modify the contents of loaded Java classes. If we store such modified classes into the AOT cache, their contents will no longer match the original class files (from application JAR files, etc). As a result, when using the AOT cache in production runs, the application may have unexpected behavior.
> 
> With this PR, we allow JVMTI agents in the AOT workflow. To address the above concern, we ensure that JVMTI agents cannot affect the contents of AOT cache:
> 
> -  In training runs (`java -XX:AOTMode=record`), JVMTI agents are allowed, but the AOT configuration file should filter out classes that are transformed by the agents. This can be checking `InstanceKlass::has_been_redefined()` and `ClassFileStream::from_class_file_load_hook()`.
> 
> - In the assembly phase (`java -XX:AOTMode=record`), agents can be specified in the command-line. However, since the assembly phase doesn't execute any application logic, we will also not load any of the specified agents. Therefore, the agents cannot affect the contents of the AOT cache created in the assembly phase.

src/hotspot/share/prims/jvmtiAgentList.cpp line 79:

> 77:   JvmtiAgent* next = head(list);
> 78: #if INCLUDE_CDS
> 79:   if (_disable_agent_list) {

Instead of intercepting here would it be better if we can clear `JvmtiAgentList::_list` in `CDSConfig::check_aotmode_create()`?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/25170#discussion_r2087603909