FFM API allocation shootout

[Chris Vest]

On Tue, Aug 1, 2023 at 5:55 PM Maurizio Cimadamore <
maurizio.cimadamore at oracle.com> wrote:

>
> I mean, I take it that, if it's a choice between performance and
> tracking, most developers who chimed in seem all to be converging
> towards disentangling Arenas from Bits::reserveMemory (so that we can
> get better allocation performance).
>
> But I'd like to ask a control question: let's assume that
> Bits::reserveMemory was free (or that we found some way to make it much
> less expensive), would opinion then shift towards having a single JDK
> parameter to control all off-heap memory allocation?
>

Is it also free when the limit has been reached, though? A limit needs to
be able to prevent allocations from going through, either by waiting or
producing and error.
Systems may have multiple different pools of memory beyond what is built
into the JVM, each with their separate limits.
Such limits are better applied in the application.
For instance, in Neo4j there are 3 main consumers of off-heap memory: the
page cache, Lucene, and Netty, and each of these should ideally not be able
to compromise the stability of the others.
Having a JVM-wide limit is counter to this, and I know that at least two of
the mentioned components take action to mitigate this.


>
> Or is it still the case that, as Arena can be customized, custom arenas
> might decide to track memory how they wish, and we should not bias the
> API against this or that particular way of tracking memory usage ?
>

I see some parallels to the Executors, who are also life-cycled
repositories of limited resources.
It has become customary for libraries and applications to offer ways to
configure or pass in the executors they need, as a way for people to
control resource usage, while at the same time the default Executor (as
much as newCachedThreadPool() can be considered a default) place no limits.
Customizing Arenas, through combinators or otherwise, could be a way to do
the same for placing controls and limits on off-heap memory.

Tracking (as in, measuring usage), on the other hand, makes sense to have
built-in.
When operating systems in large deployments, native memory usage is one of
the metrics you always want to collect and monitor.
Having every library and framework report their off-heap memory usage in
different ways can be useful for drilling into problems, but you also want
the sum of usage to be track in a well-known place.
I don't have a strong opinion on this being one metric for ByteBuffer and
MemorySegments combined, or if they are exposed as two separate well-known
metrics; you just don't want it to be a dozen
mostly-library-and-app-defined metrics.


>
> Cheers
> Maurizio
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/panama-dev/attachments/20230802/53031011/attachment-0001.htm>