[crac] RFR: 8364156: [CRaC] Recompile methods decompiled during or shortly after C/R

[Timofei Pushkin]

On Mon, 28 Jul 2025 07:04:26 GMT, Timofei Pushkin <tpushkin at openjdk.org> wrote:

> Implements recording and triggering recompilation of methods decompiled during and shortly after C/R. The length of the "shortly-after" period can be controlled by the user via `-Djdk.crac.recompilation-delay-ms=<long>`, the default is 10ms.
> 
> Measuring the after-restore regression before and after this change shows improvement on hello-world apps. The numbers are the regression of the first after-restore 1 sec iteration vs average over 5 last before-checkpoint 1 sec iterations, i.e. lower means better; mean ± std dev over 30 runs. Measured in a Linux VM on MacOS, with CRIU, with 5ms after-restore recompilation delay.
> 
> |                          | Before      | After       |
> |--------------------------|-------------|-------------|
> | Helidon hello-world      | 0.9 ± 1.2 % | 0.4 ± 1.0 % |
> | Spring boot CRaC example | 9.5 ± 2.4 % | 6.4 ± 2.5 % |
> 
> Currently decompilations are recorded during C/R + during the after-restore delay and then they all get recompiled together. An enhancement planned in the future is to record only during C/R recompiling immediately after restore and during the after-restore delay recompile immediately without recording. This way the recompilations would finish faster, also setting `jdk.crac.recompilation-delay-ms` to a higher value would have less negative impact.
> 
> Implementation notes:
> - Native code (`nmethod`) is considered "decompiled" when `make_not_entrant(...)` is called on it — a new mandatory parameter is added to the signature of this method that controls whether CRaC should consider this particular decompilation for recompilation or not.
>   - The parameter is needed because we do not want to recompile, for example, when `nmethod` is thrown away once a replacement for it has been compiled (e.g. on another compilation level).
>   - The parameter has no default so that we get compilation errors when new calls to `make_not_entrant(...)` are introduced in the mainline — we will be deciding whether these new calls should be considered for recompilation or not. If CRaC was in the mainline I would enable the recompilation by default.
> - The new native `CRaCRecorder` class is the core of the implementation — it records the decompilations and handles their recompilation.
>   - `jdk.crac.internal.mirror.Core` notifies it when to start and stop the recording, it also handles the after-restore recompilation delay.
>   - `nmethod.make_not_entrant(...)` notifies it about decompiled code.

src/hotspot/share/runtime/cracRecompiler.cpp line 171:

> 169:     log_debug(crac)("Finishing recording decompilations and requesting %i recompilations", decomps->length());
> 170:     // fence should allow the recorder threads to stop locking quicker
> 171:     Atomic::release_store_fence(&decompilations, static_cast<decltype(decompilations)>(nullptr));

I've realized that after this nothing is guarding the recorded methods from being deleted by `RedefineClasses`. Need to fix this before integrating.

-------------

PR Review Comment: https://git.openjdk.org/crac/pull/251#discussion_r2235008362