<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Hi Matthias,</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Thank you for your mail!<br>
<br>
We are working on a seemingly very related issue where we want to reuse a <span style="font-family: "Aptos Mono", Aptos_EmbeddedFont, Aptos_MSFontService, monospace;">
MemorySegment</span> for "system calls" (e.g. <span style="font-family: "Aptos Mono", Aptos_EmbeddedFont, Aptos_MSFontService, monospace;">
open()</span> and<span style="font-family: "Aptos Mono", Aptos_EmbeddedFont, Aptos_MSFontService, monospace;"> socket()</span>) that is capturing the error number (in the reused
<span style="font-family: "Aptos Mono", Aptos_EmbeddedFont, Aptos_MSFontService, monospace;">
MemorySegment</span>). We are seeing similar performance improvements as you have identified. Here, we are careful to make sure that we cover the case when a virtual thread is unmounted from its carrier during a potentially lengthy system call and another
virtual thread is mounted on the original platform thread and subsequently invokes another system call.<br>
<br>
</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Here is the PR: <a href="https://github.com/openjdk/jdk/pull/22391" id="LPlnk164947" class="OWAAutoLink">
https://github.com/openjdk/jdk/pull/22391</a></div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
Also, we are exploring having a perhaps even more performant reuse constructs compared to using
<span style="font-family: "Aptos Mono", Aptos_EmbeddedFont, Aptos_MSFontService, monospace;">
TerminatingThreadLocal</span>. This may or may not be included in the JDK down the line.<br>
<br>
Please take a look at the PR above and feel free to comment there to.<br>
<br>
I think there are synergies to harvest here.</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Best, Per</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div id="appendonsend"></div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> panama-dev <panama-dev-retn@openjdk.org> on behalf of Matthias Ernst <matthias@mernst.org><br>
<b>Sent:</b> Thursday, January 16, 2025 11:00 AM<br>
<b>To:</b> panama-dev@openjdk.org <panama-dev@openjdk.org><br>
<b>Subject:</b> [FFM performance] Intermediate buffer allocation when returning structs</font>
<div> </div>
</div>
<div>
<div dir="ltr">
<div dir="ltr">Hi, I noticed a source of overhead when calling foreign functions with small aggregate return values.<br>
<br>
<div>For example, a function returning a <font face="monospace">struct Vector2D { double x; double y }</font> will cause a malloc/free inside the downcall handle on every call. On my machine, this accounts for about 80% of the call overhead.</div>
<div><br>
</div>
<div>Choice stack:</div>
<div>
<pre style="box-sizing:border-box; font-size:11.9px; margin-top:0px; overflow:auto; line-height:1.45; color:rgb(31,35,40); border-radius:6px"><code style="box-sizing:border-box; padding:0px; margin:0px; background:transparent; border-radius:6px; word-break:normal; border:0px; display:inline; overflow:visible; line-height:inherit"> java.lang.Thread.State: RUNNABLE
<b> at jdk.internal.misc.Unsafe.allocateMemory0(java.base@25-ea/Native Method)
</b>...
<b> at jdk.internal.foreign.abi.SharedUtils.newBoundedArena(java.base@25-ea/SharedUtils.java:386)
</b> at jdk.internal.foreign.abi.DowncallStub/0x000001f001084c00.invoke(java.base@25-ea/Unknown Source)
at java.lang.invoke.DirectMethodHandle$Holder.invokeStatic(java.base@25-ea/DirectMethodHandle$Holder)
</code></pre>
</div>
<div>While it might be difficult to eliminate these intermediate buffers, I would propose to try reusing them.</div>
<div><br>
</div>
<div>
<div>
<div>What's happening here:</div>
</div>
</div>
<div>* the ARM64 ABI returns such a struct in two 128 bit registers v0/v1 [0]</div>
<div>* the VM stub calling convention around this expects an output buffer to copy v0/v1 into: [1]</div>
<div><font face="monospace">stub(out) { ... out[0..16) = v0; out[16..32) = v1; }</font></div>
<div>* the FFM downcall calling convention OTOH expects a user-provided SegmentAllocator to allocate a 16 byte StructLayout(JAVA_DOUBLE, JAVA_DOUBLE). The generated method handle to adapt to the stub looks roughly like this [2]:</div>
<div> ffm(allocator) {</div>
<div><b> tmp = malloc(32)</b></div>
<div> stub(tmp)</div>
<div> result = allocator.allocate(16)</div>
<div> result[0..8) = tmp[0..8)</div>
<div> result[8..16) = tmp[16..24)</div>
<b> free(tmp)</b></div>
<div dir="ltr"> return result<br class="x_gmail-Apple-interchange-newline">
<div>}</div>
<div><br>
</div>
<div>Now there's an easy way around this for the user by using a different native signature:</div>
<div>
<div><font face="monospace">void g(Vector2D *out) { *out = f(); }</font></div>
<div>This eliminates the intermediate buffer altogether.</div>
<div><br>
</div>
<div>
<div>However, if we wanted to optimize the return-by-value path, I can think of three options:</div>
<div>* enhance the stub calling conventions to directly copy only the narrowed output registers into the result buffer. This looks rather involved.</div>
<div>* allocate the tmp buffer using the user's allocator as well (e.g. in conjunction with the result + slicing). The Linker api is somewhat lenient about how `allocator` will be exactly invoked: "used by the linker runtime to allocate the memory region associated
with the struct returned by the downcall method handle". However, this may be surprising to the caller.</div>
<div>* keep the tmp buffer allocation internal, but optimize it. This is what I'm proposing here.</div>
<div><br>
</div>
</div>
<div>A possible counter-argument could be "this is just one allocation out of two". However, the user has control over `allocator`, and may re-use the same segment across calls, but they have no control over the tmp allocation.</div>
<div><br>
</div>
</div>
<div>
<div>I've worked on a patch that takes this last route, using a one-element thread-local cache: <a href="https://github.com/openjdk/jdk/pull/23142">https://github.com/openjdk/jdk/pull/23142</a>, it reduces call time from 36->8ns / op on my machine and I observe
no more GC's.</div>
</div>
<div><br>
</div>
<div>Would there be interest in pursuing this?</div>
<div><br>
</div>
<div>Thx</div>
<div>Matthias</div>
<div><br>
</div>
<div><br>
</div>
<div>[0] <a href="https://learn.microsoft.com/en-us/cpp/build/arm64-windows-abi-conventions?view=msvc-170#return-values" target="_blank">https://learn.microsoft.com/en-us/cpp/build/arm64-windows-abi-conventions?view=msvc-170#return-values</a></div>
<div>[1] <a href="https://github.com/openjdk/jdk/blob/9c430c92257739730155df05f340fe144fd24098/src/java.base/share/classes/jdk/internal/foreign/abi/CallingSequenceBuilder.java#L97" target="_blank">https://github.com/openjdk/jdk/blob/9c430c92257739730155df05f340fe144fd24098/src/java.base/share/classes/jdk/internal/foreign/abi/CallingSequenceBuilder.java#L97</a></div>
<div>[2] "binding context": <a href="https://github.com/openjdk/jdk/blob/9c430c92257739730155df05f340fe144fd24098/src/java.base/share/classes/jdk/internal/foreign/abi/BindingSpecializer.java#L296">https://github.com/openjdk/jdk/blob/9c430c92257739730155df05f340fe144fd24098/src/java.base/share/classes/jdk/internal/foreign/abi/BindingSpecializer.java#L296</a></div>
</div>
</div>
</div>
</body>
</html>