FFM API allocation shootout

[Maurizio Cimadamore]

On 02/08/2023 17:20, Chris Vest wrote:
>
>
> On Tue, Aug 1, 2023 at 5:55 PM Maurizio Cimadamore 
> <maurizio.cimadamore at oracle.com> wrote:
>
>
>     I mean, I take it that, if it's a choice between performance and
>     tracking, most developers who chimed in seem all to be converging
>     towards disentangling Arenas from Bits::reserveMemory (so that we can
>     get better allocation performance).
>
>     But I'd like to ask a control question: let's assume that
>     Bits::reserveMemory was free (or that we found some way to make it
>     much
>     less expensive), would opinion then shift towards having a single JDK
>     parameter to control all off-heap memory allocation?
>
>
> Is it also free when the limit has been reached, though? A limit needs 
> to be able to prevent allocations from going through, either by 
> waiting or producing and error.
> Systems may have multiple different pools of memory beyond what is 
> built into the JVM, each with their separate limits.
> Such limits are better applied in the application.
> For instance, in Neo4j there are 3 main consumers of off-heap memory: 
> the page cache, Lucene, and Netty, and each of these should ideally 
> not be able to compromise the stability of the others.
> Having a JVM-wide limit is counter to this, and I know that at least 
> two of the mentioned components take action to mitigate this.

I tend to agree with what you say. As explained, the direct memory limit 
is a strange combination of "tracking memory usage", but one that also 
has an active component - in the sense that when the limit is reached, 
some "magic" happens to try and see if cleaners might try to execute and 
free some more memory.

While we can't get rid of this logic for ByteBuffer (and, likely, also 
for segments allocated in automatic arenas), there are strong arguments 
in favor of not tracking memory usage and/or enforcing limits if you use 
explicit deallocation with FFM API (after all, clients that do this, 
want to be more in control).

>
>     Or is it still the case that, as Arena can be customized, custom
>     arenas
>     might decide to track memory how they wish, and we should not bias
>     the
>     API against this or that particular way of tracking memory usage ?
>
>
> I see some parallels to the Executors, who are also life-cycled 
> repositories of limited resources.
> It has become customary for libraries and applications to offer ways 
> to configure or pass in the executors they need, as a way for people 
> to control resource usage, while at the same time the default Executor 
> (as much as newCachedThreadPool() can be considered a default) place 
> no limits.
> Customizing Arenas, through combinators or otherwise, could be a way 
> to do the same for placing controls and limits on off-heap memory.
Yes - and a better way as well (in the sense that the cost of such 
tracking can be right-sized depending on the application needs).
>
> Tracking (as in, measuring usage), on the other hand, makes sense to 
> have built-in.
> When operating systems in large deployments, native memory usage is 
> one of the metrics you always want to collect and monitor.
> Having every library and framework report their off-heap memory usage 
> in different ways can be useful for drilling into problems, but you 
> also want the sum of usage to be track in a well-known place.
> I don't have a strong opinion on this being one metric for ByteBuffer 
> and MemorySegments combined, or if they are exposed as two separate 
> well-known metrics; you just don't want it to be a dozen 
> mostly-library-and-app-defined metrics.

Such a simple counter could be realized efficiently using a LongAdder 
that is increased for each newly allocated MS/BB.

But, connecting this to what you were saying before, even calling 
LongAdder::add has a cost - if the application is already tracking 
memory its own way, one might question as to why we're "wasting time" 
updating this other counter that the app doesn't really care about?

Also, how should such counter be exposed? ByteBuffer go down the JMX 
path. But since the introduction of ByteBuffer, we now also have JFR. Do 
people expect to track these kind of things using JMX, or using JFR events?

Maurizio

>
>     Cheers
>     Maurizio
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/panama-dev/attachments/20230802/9219cc04/attachment.htm>