RFR: 8318586: Explicitly handle upcall stub allocation failure

[Jorn Vernee]

On Wed, 25 Oct 2023 09:16:25 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

> I find it pretty weird to terminate the VM if we cannot allocate upcall stub. Does this mean the user code could actually terminate the VM on this `fatal`? Unit test suggests so.
> 
> Can the VM code actually handle things without upcall stub present, if e.g. memory is exhausted?

I think the question is whether the user can do anything reasonable if the allocation fails. Upcall stubs are allocated as a result of a call to `Linker.upcallStub`. That means that, one way or another we can not satisfy a direct user request, and the allocation failure would be visible. Whether that is through a fatal error, or some kind of exception, I'm not sure.

But it sounds like you're saying that plain user code should never result in a VM error (if we can help it). That is something I agree with. We'd have to throw some exception from `Linker.upcallStub` if the allocation fails (not sure of OOME is the right one for CodeCache exhaustion). And probably the same for `Linker.downcallLinker`.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/16311#issuecomment-1778890991