RFR: 8318586: Explicitly handle upcall stub allocation failure

[Jorn Vernee]

On Wed, 25 Oct 2023 09:47:13 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

> > But it sounds like you're saying that plain user code should never result in a VM error (if we can help it).
> 
> Yes, exactly. Granted, there are resource exhaustion situations where the overall progress could be sluggish (e.g. if we near the Java heap OOME), but we don't usually elevate that to globally shutting down the JVM, unless user explicitly requests this, e.g. with `-XX:+ExitOnOutOfMemoryError`.
> 
> I think the situation for `RuntimeStub`-s is a bit different, as we expect to have more or less constant number of them, mostly allocated upfront. Failing to allocate `RuntimeStub` then looks like a configuration issue. But for `UpcallStub`-s -- correct me if I am wrong here -- we can have unbounded number of them, right? Which exposes us to globally visible VM failure if there is a misbehaving code.

FWIW, we use RuntimeStub for downcall stubs allocated by the linker. So there is also an unbounded number of those. (Well technically bounded by the 255 argument limit * all argument type combinations, but that number is very large).

> It is not the problem with this concrete PR, which I think is fine, but it exposes the larger, more important architectural question.

Ok, I'll discuss with the others in the FFM team. I think if we turn this failure into an OOME (leaning on that side at the moment), then we also need a spec change + CSR.

I'll wait with this PR until we reach some conclusion.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/16311#issuecomment-1778912083