RFR: 8364501: Compiler shutdown crashes on access to deleted CompileTask

[Aleksey Shipilev]

On Fri, 8 Aug 2025 12:30:36 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

> See the bug for more investigation. 
> 
> In short, with recent changes to `delete` `CompileTask`-s, we end up in the rare situation where we can access tasks that have been already deleted. The major and obivous mistake I committed myself with [JDK-8361752](https://bugs.openjdk.org/browse/JDK-8361752) in `CompileQueue::delete_all`: the code first `delete`-s, then asks for `next` (facepalms). 
> 
> Another case is less trivial, and mostly fix in abundance of caution: in `wait_for_completion`, we can exit while blocking task is still in queue. Current code skip deletions only when compiler is shutdown for compilation, but I think the condition should be stronger: unless the task is completed, we should assume it might carry the queue-ing `next`/`prev` pointers that `delete_all` would need, and skip deletion. Realistically, it would "leak" only on compiler shutdown, like before.
> 
> I have also put in some diagnostic code to catch the lifecycle issues like this more reliably, and cleaned up `next`, `prev` lifecycle to clearly disconnect the `CompileTasks` that are no longer in queue.
> 
> Additional testing:
>  - [x] Linux AArch64 server fastdebug, reproducer no longer fails
>  - [x] Linux AArch64 server fastdebug, `compiler`
>  - [x] Linux AArch64 server fastdebug, `all`

Looking at the patch before integration led me to a more paranoid path: chickening out of `memset`-ing the C++ object in destructor. For several reasons:
 1. There is no vtable in `CompileTask` _now_, but nothing prevents us from adding it later.  Calling `memset` from subclass/virtual destructor might be a cause of fun bugs then. We do this `memset` in some places in Hotspot, but only for `struct`-looking classes.
 2. The newly added clearing of `next`/`prev` on dequeue-ing prevents most of the accidents of walking to random memory, and also makes explicit zapping not that useful to reproduce bugs.
 3. I think we should look into a more generic zapping for all `CHeapObj`-s, when we know it is safe (e.g. right before calling `free` to native allocator). This is tracked by [JDK-8365165](https://bugs.openjdk.org/browse/JDK-8365165).

This does not invalidate testing, since I just removed the `ASSERT` block. So I can integrate after someone re-approves :)

-------------

PR Comment: https://git.openjdk.org/jdk/pull/26696#issuecomment-3175697792