RFR: 8324241: Always record evol_method deps to avoid excessive method flushing

[Evgeny Astigeevich]

On Mon, 22 Jan 2024 16:37:14 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> Currently we don't record dependencies on redefined methods (i.e. `evol_method` dependencies) in JIT compiled methods if none of the `can_redefine_classes`, `can_retransform_classes` or `can_generate_breakpoint_events` JVMTI capabalities is set. This means that if a JVMTI agent which requests one of these capabilities is dynamically attached, all the methods which have been JIT compiled until that point, will be marked for deoptimization and flushed from the code cache. For large, warmed-up applications this mean deoptimization and instant recompilation of thousands if not then-thousands of methods, which can lead to dramatic performance/latency drop-downs for several minutes.
>> 
>> One could argue that dynamic agent attach is now deprecated anyway (see [JEP 451: Prepare to Disallow the Dynamic Loading of Agents](https://openjdk.org/jeps/451)) and this problem could be solved by making the recording of `evol_method` dependencies dependent on the new `-XX:+EnableDynamicAgentLoading` flag isntead of the concrete JVMTI capabilities (because the presence of the flag indicates that an agent will be loaded eventually).
>> 
>> But there a single, however important exception to this rule and that's JFR. JFR is advertised as low overhead profiler which can be enabled in production at any time. However, when JFR is started dynamically (e.g. through JCMD or JMX) it will silently load a HotSpot internl JVMTI agent which requests the `can_retransform_classes` and retransforms some classes. This will inevitably trigger the deoptimization of all compiled methods as described above.
>> 
>> I'd therefor like to propose to *always* and unconditionally record `evol_method` dependencies in JIT compiled code by exporting the relevant properties right at startup in `init_globals()`:
>> ```c++
>>  jint init_globals() {
>>    management_init();
>>    JvmtiExport::initialize_oop_storage();
>> +#if INCLUDE_JVMTI
>> +  JvmtiExport::set_can_hotswap_or_post_breakpoint(true);
>> +  JvmtiExport::set_all_dependencies_are_recorded(true);
>> +#endif
>> 
>> 
>> My measurements indicate that the overhead of doing so is minimal (around 1% increase of nmethod size) and justifies the benefit. E.g. a Spring Petclinic application started with `-Xbatch -XX:+UnlockDiagnosticVMOptions -XX:+LogCompilation` compiles about ~11500 methods (~9000 with C1 and ~2500 with C2) resulting in an aggregated nmethod size of around ~40bm. Additionally recording `evol_method` dependencies only increases this size be about 400kb....
>
>> Instead of unconditionally recording evol_method dependencies we could guard the recording by a new flag. But this would only make sense if that flag would be on by default and I don't know if such a flag is justified just for the rare (or non-existent?) cases where somebody wants to disable the recording of the dependencies.
> 
> I think introducing a diagnostic flag is sensible here. If we figure out much later that this solution comes with some other (worse) problems, the diagnostic flag gives us the options to: a) clearly point at this addition as the culprit; b) have the easily deployable solution to restore the original behavior.
> 
> For the change itself, we need to amend the comment near `VM_RedefineClasses::flush_dependent_code` definition that talks about this peculiar behavior, which now changes. Actually, maybe even the implementation of `flush_dependent_code` should now trust (and assert) that all dependencies are now recorded?

@shipilev I filed https://bugs.openjdk.org/browse/JDK-8324318 which is related to this change.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/17509#issuecomment-1904402169