<!DOCTYPE html><html><head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
  </head>
  <body>
    <p>Hi,<br>
      yes, the API has changed considerably when it comes to managing
      the lifetime of pointers that have been allocated extrenally (we
      sometimes refer to this use case as "external allocation").</p>
    <p>This is covered in this document in more details:</p>
    <p><a class="moz-txt-link-freetext" href="https://cr.openjdk.org/~mcimadamore/panama/why_lifetimes.html">https://cr.openjdk.org/~mcimadamore/panama/why_lifetimes.html</a></p>
    <p>And there's few slides on this topic in my latest JVMLS talk:</p>
    <p><a class="moz-txt-link-freetext" href="https://youtu.be/kUFysMkMS00?feature=shared&t=1718">https://youtu.be/kUFysMkMS00?feature=shared&t=1718</a></p>
    <p>The main idea is that pointers returned by native libraries are
      just shallow memory segments that have no size and a global scope
      (we call such segments "zero-length memory segments" [1]). In
      order to use these segments correctly, one must give them a size
      (otherwise dereference operations will fail). Then, there are
      typically two cases:</p>
    <p>* the library (callee) is responsible for deallocation - in such
      case, no other operation (other than the resize) is needed;<br>
      * the client (caller) is responsible for deallocation - in such
      case, when resizing the segment it is typically useful to also
      provide a suitable Arena (which determines _when_ the segment will
      be deallocated) and a cleanup actiojn (which determines _how_ the
      segment will be deallocated).</p>
    <p>In both these case, the unsafe MemorySegment::reinterpret method
      has to be used, which means that the "--enable-native-access" flag
      must be passed on the command line (to avoid the warnings). As a
      result of these changes, we have now consolidated other operations
      such as turning a long address into a memory segment (`MemorySegment::ofAddress`,
      which now simply returns a zero-length memory segment [2]).</p>
    <p>Maurizio<br>
    </p>
    <p>[1] -
<a class="moz-txt-link-freetext" href="https://download.java.net/java/early_access/jdk22/docs/api/java.base/java/lang/foreign/MemorySegment.html#wrapping-addresses">https://download.java.net/java/early_access/jdk22/docs/api/java.base/java/lang/foreign/MemorySegment.html#wrapping-addresses</a><br>
      [2] -
<a class="moz-txt-link-freetext" href="https://download.java.net/java/early_access/jdk22/docs/api/java.base/java/lang/foreign/MemorySegment.html#ofAddress(long)">https://download.java.net/java/early_access/jdk22/docs/api/java.base/java/lang/foreign/MemorySegment.html#ofAddress(long)</a><br>
    </p>
    <div class="moz-cite-prefix">On 23/01/2024 01:11, Red IO wrote:<br>
    </div>
    <blockquote type="cite" cite="mid:CABKyW1spL2AvY1YN5VhGyHCGEENaonT3U8VGUZh8zDDcKvTg9g@mail.gmail.com">
      
      <div dir="auto">Thanks Maurizio for the clear explanation.
        <div dir="auto">Since last time I experimented with Panama the
          api changed a lot.
          <div dir="auto">I will need to look more into what jextract
            generates to understand the current api better. </div>
          <div dir="auto"><br>
          </div>
          <div dir="auto">Great regards </div>
          <div dir="auto">RedIODev </div>
        </div>
      </div>
      <br>
      <div class="gmail_quote">
        <div dir="ltr" class="gmail_attr">On Mon, Jan 22, 2024, 22:42
          Maurizio Cimadamore <<a href="mailto:maurizio.cimadamore@oracle.com" moz-do-not-send="true" class="moz-txt-link-freetext">maurizio.cimadamore@oracle.com</a>>
          wrote:<br>
        </div>
        <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
          <div>
            <p><br>
            </p>
            <div>On 22/01/2024 20:01, Red IO wrote:<br>
            </div>
            <blockquote type="cite">
              <div dir="auto">I know that the original question is
                answered, but how would you free memory in case the
                libraries documentation says to use free. Would you just
                create a method handle to libc free and call it with the
                pointer or is there an api to free native memory?</div>
            </blockquote>
            <p>If the API says to use free, then you should have a
              downcall method handle for "free" and call that (using the
              same technique with MemorySegment::reinterpret I showed in
              the last email).</p>
            <p>(Note: when using jextract, chances are that "free" will
              come along for the ride in the extracted classes).<br>
            </p>
            <p>Maurizio<br>
            </p>
            <blockquote type="cite">
              <div dir="auto">
                <div dir="auto">Great regards </div>
                <div dir="auto">RedIODev </div>
              </div>
              <br>
              <div class="gmail_quote">
                <div dir="ltr" class="gmail_attr">On Mon, Jan 22, 2024,
                  19:36 Maurizio Cimadamore <<a href="mailto:maurizio.cimadamore@oracle.com" target="_blank" rel="noreferrer" moz-do-not-send="true" class="moz-txt-link-freetext">maurizio.cimadamore@oracle.com</a>>
                  wrote:<br>
                </div>
                <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi
                  tison,<br>
                  in this case, the function is returning a pointer.
                  What happens to that <br>
                  pointer is 100% library-dependent. In this case, doing
                  `man getpwnam` <br>
                  gives us some clues:<br>
                  <br>
                  > The return value may point to a static area, and
                  may be overwritten  by<br>
                  >        subsequent  calls  to  getpwent(3),
                  getpwnam(), or getpwuid().  <br>
                  > (Do not<br>
                  >        pass the returned pointer to free(3).)<br>
                  <br>
                  Basically, the returned pointer points to an area that
                  is managed by the <br>
                  library, so you don't have to free.<br>
                  <br>
                  This means the code you have seems correct to me.<br>
                  <br>
                  Other libraries will behave differently, for instance
                  they might have a <br>
                  pair of functions such as "createFoo" and
                  "destroyFoo", the former <br>
                  allocating region of memory, and the latter destroying
                  such region. In <br>
                  such cases it might be helpful to associate the return
                  memory segment to <br>
                  an existing arena *and* also attach a cleanup function
                  (via <br>
                  MemorySegment::reinterpret), so that, when the arena
                  is closed, <br>
                  "destroyFoo" is called on the pointer.<br>
                  <br>
                  Hope this helps<br>
                  <br>
                  Maurizio<br>
                  <br>
                  <br>
                  On 22/01/2024 18:28, tison wrote:<br>
                  > Here is the code snippet:<br>
                  ><br>
                  >      public static void main(String[] args)
                  throws Throwable {<br>
                  >          final Linker linker =
                  Linker.nativeLinker();<br>
                  >          final SymbolLookup libc =
                  linker.defaultLookup();<br>
                  >          final MethodHandle handle =<br>
                  >
                  linker.downcallHandle(libc.find("getpwnam").orElseThrow(),<br>
                  > FunctionDescriptor.of(ValueLayout.ADDRESS,
                  ValueLayout.ADDRESS));<br>
                  >          try (Arena arena = Arena.ofConfined()) {<br>
                  >              final MemorySegment passwd =
                  (MemorySegment)<br>
                  > handle.invoke(arena.allocateUtf8String("tison"));<br>
                  >              System.out.println("passwd=" +
                  passwd);<br>
                  >              System.out.println("pw_name=" +<br>
                  >
                  passwd.reinterpret(Long.MAX_VALUE).get(ValueLayout.ADDRESS,<br>
                  >
                  48).reinterpret(Long.MAX_VALUE).getUtf8String(0));<br>
                  >          }<br>
                  >      }<br>
                  ><br>
                  > I wonder if I should explicitly free passwd that
                  is returned from a<br>
                  > native method. If so, how? If not, how the Arena
                  tracks it and free on<br>
                  > closed?<br>
                  ><br>
                  > Best,<br>
                  > tison.<br>
                </blockquote>
              </div>
            </blockquote>
          </div>
        </blockquote>
      </div>
    </blockquote>
  </body>
</html>