<!DOCTYPE html><html><head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body>
<p>Hi,<br>
yes, the API has changed considerably when it comes to managing
the lifetime of pointers that have been allocated extrenally (we
sometimes refer to this use case as "external allocation").</p>
<p>This is covered in this document in more details:</p>
<p><a class="moz-txt-link-freetext" href="https://cr.openjdk.org/~mcimadamore/panama/why_lifetimes.html">https://cr.openjdk.org/~mcimadamore/panama/why_lifetimes.html</a></p>
<p>And there's few slides on this topic in my latest JVMLS talk:</p>
<p><a class="moz-txt-link-freetext" href="https://youtu.be/kUFysMkMS00?feature=shared&t=1718">https://youtu.be/kUFysMkMS00?feature=shared&t=1718</a></p>
<p>The main idea is that pointers returned by native libraries are
just shallow memory segments that have no size and a global scope
(we call such segments "zero-length memory segments" [1]). In
order to use these segments correctly, one must give them a size
(otherwise dereference operations will fail). Then, there are
typically two cases:</p>
<p>* the library (callee) is responsible for deallocation - in such
case, no other operation (other than the resize) is needed;<br>
* the client (caller) is responsible for deallocation - in such
case, when resizing the segment it is typically useful to also
provide a suitable Arena (which determines _when_ the segment will
be deallocated) and a cleanup actiojn (which determines _how_ the
segment will be deallocated).</p>
<p>In both these case, the unsafe MemorySegment::reinterpret method
has to be used, which means that the "--enable-native-access" flag
must be passed on the command line (to avoid the warnings). As a
result of these changes, we have now consolidated other operations
such as turning a long address into a memory segment (`MemorySegment::ofAddress`,
which now simply returns a zero-length memory segment [2]).</p>
<p>Maurizio<br>
</p>
<p>[1] -
<a class="moz-txt-link-freetext" href="https://download.java.net/java/early_access/jdk22/docs/api/java.base/java/lang/foreign/MemorySegment.html#wrapping-addresses">https://download.java.net/java/early_access/jdk22/docs/api/java.base/java/lang/foreign/MemorySegment.html#wrapping-addresses</a><br>
[2] -
<a class="moz-txt-link-freetext" href="https://download.java.net/java/early_access/jdk22/docs/api/java.base/java/lang/foreign/MemorySegment.html#ofAddress(long)">https://download.java.net/java/early_access/jdk22/docs/api/java.base/java/lang/foreign/MemorySegment.html#ofAddress(long)</a><br>
</p>
<div class="moz-cite-prefix">On 23/01/2024 01:11, Red IO wrote:<br>
</div>
<blockquote type="cite" cite="mid:CABKyW1spL2AvY1YN5VhGyHCGEENaonT3U8VGUZh8zDDcKvTg9g@mail.gmail.com">
<div dir="auto">Thanks Maurizio for the clear explanation.
<div dir="auto">Since last time I experimented with Panama the
api changed a lot.
<div dir="auto">I will need to look more into what jextract
generates to understand the current api better. </div>
<div dir="auto"><br>
</div>
<div dir="auto">Great regards </div>
<div dir="auto">RedIODev </div>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Mon, Jan 22, 2024, 22:42
Maurizio Cimadamore <<a href="mailto:maurizio.cimadamore@oracle.com" moz-do-not-send="true" class="moz-txt-link-freetext">maurizio.cimadamore@oracle.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>
<p><br>
</p>
<div>On 22/01/2024 20:01, Red IO wrote:<br>
</div>
<blockquote type="cite">
<div dir="auto">I know that the original question is
answered, but how would you free memory in case the
libraries documentation says to use free. Would you just
create a method handle to libc free and call it with the
pointer or is there an api to free native memory?</div>
</blockquote>
<p>If the API says to use free, then you should have a
downcall method handle for "free" and call that (using the
same technique with MemorySegment::reinterpret I showed in
the last email).</p>
<p>(Note: when using jextract, chances are that "free" will
come along for the ride in the extracted classes).<br>
</p>
<p>Maurizio<br>
</p>
<blockquote type="cite">
<div dir="auto">
<div dir="auto">Great regards </div>
<div dir="auto">RedIODev </div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Mon, Jan 22, 2024,
19:36 Maurizio Cimadamore <<a href="mailto:maurizio.cimadamore@oracle.com" target="_blank" rel="noreferrer" moz-do-not-send="true" class="moz-txt-link-freetext">maurizio.cimadamore@oracle.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi
tison,<br>
in this case, the function is returning a pointer.
What happens to that <br>
pointer is 100% library-dependent. In this case, doing
`man getpwnam` <br>
gives us some clues:<br>
<br>
> The return value may point to a static area, and
may be overwritten by<br>
> subsequent calls to getpwent(3),
getpwnam(), or getpwuid(). <br>
> (Do not<br>
> pass the returned pointer to free(3).)<br>
<br>
Basically, the returned pointer points to an area that
is managed by the <br>
library, so you don't have to free.<br>
<br>
This means the code you have seems correct to me.<br>
<br>
Other libraries will behave differently, for instance
they might have a <br>
pair of functions such as "createFoo" and
"destroyFoo", the former <br>
allocating region of memory, and the latter destroying
such region. In <br>
such cases it might be helpful to associate the return
memory segment to <br>
an existing arena *and* also attach a cleanup function
(via <br>
MemorySegment::reinterpret), so that, when the arena
is closed, <br>
"destroyFoo" is called on the pointer.<br>
<br>
Hope this helps<br>
<br>
Maurizio<br>
<br>
<br>
On 22/01/2024 18:28, tison wrote:<br>
> Here is the code snippet:<br>
><br>
> public static void main(String[] args)
throws Throwable {<br>
> final Linker linker =
Linker.nativeLinker();<br>
> final SymbolLookup libc =
linker.defaultLookup();<br>
> final MethodHandle handle =<br>
>
linker.downcallHandle(libc.find("getpwnam").orElseThrow(),<br>
> FunctionDescriptor.of(ValueLayout.ADDRESS,
ValueLayout.ADDRESS));<br>
> try (Arena arena = Arena.ofConfined()) {<br>
> final MemorySegment passwd =
(MemorySegment)<br>
> handle.invoke(arena.allocateUtf8String("tison"));<br>
> System.out.println("passwd=" +
passwd);<br>
> System.out.println("pw_name=" +<br>
>
passwd.reinterpret(Long.MAX_VALUE).get(ValueLayout.ADDRESS,<br>
>
48).reinterpret(Long.MAX_VALUE).getUtf8String(0));<br>
> }<br>
> }<br>
><br>
> I wonder if I should explicitly free passwd that
is returned from a<br>
> native method. If so, how? If not, how the Arena
tracks it and free on<br>
> closed?<br>
><br>
> Best,<br>
> tison.<br>
</blockquote>
</div>
</blockquote>
</div>
</blockquote>
</div>
</blockquote>
</body>
</html>